Comments (54)
Thank you for commenting on this for a hobbyist like me who does DeepLearning as a hobby. I am not an engineer or a researcher.
Is there an easy way to convert ONNX or PB from (NCHW) to (NHWC)?
No. By the way, I've already successfully converted NCHW to NHWC, but in a very primitive way I did. Since Tensorflow's Conv2D and several other OPs do not support NCHW, this was accomplished by inserting Transpose OPs before and after each OP. While this method can be made to infer correctly, the inserted Transpose OP resulted in unnecessary overhead and a significant loss of original performance. I used a combination of Keras and OpenVINO's model_optimizer to achieve the NHWC to NCHW conversion. (Converting backwards is easy.)
Is there an easy way to convert TF1-pb to TF2-saved_models.pb ?
Yes. I've described how to do the conversion in some of my blog posts below.
[English ver.] [Tensorflow Lite] Various Neural Network Model quantization methods for Tensorflow Lite (Weight Quantization, Integer Quantization, Full Integer Quantization, Float16 Quantization, EdgeTPU). As of May 05, 2020.
Alternatively, you can find a repository of perfect tutorials below.
https://github.com/leimao/Frozen_Graph_TensorFlow
Is NHWC slowing down execution on the GPU?
I'm not sure. I'm not really interested in using a high performance GPU. I only benchmark with low performance CPUs and edge accelerators. However, I've seen blogs in the past where Japanese engineers have done comparative benchmarking between NCHW and NHWC. However, this article does not refer to the performance of reasoning, but rather shows an increase in learning speed. The improvement in learning speed appears to be from a few percent to a few dozen percent.
Japanese article: TensorFlow/Kerasでchannels_firstにするとGPUの訓練が少し速くなる話 - @koshian2
How many FPS do you get on Google Coral TPU-Edge and RaspberryPi4 for yolov4-tiny (int8)?
RaspberryPi4 + CPU only + INT8 + Tensorflow Lite (4 threads) + 256x256 with 88ms/inference Performance.
Twitter: https://twitter.com/PINTO03091/status/1283429524462428160?s=20
RaspberryPi4 + CPU only + INT8 + Tensorflow Lite (4 threads) + 416x416 with 243ms/inference Performance.
Twitter: https://twitter.com/PINTO03091/status/1282319908097810437?s=20
Unfortunately, the conversion to models for Coral TPU-Edge was not successful due to a bug in the Tensorflow Lite Converter.
What script did you use to get yolov4_tiny_voc.json?
- First, clone the following repositories.
https://github.com/bubbliiiing/yolov4-tiny-keras.git - All you have to do is modify the
test.py
script, change it to the following and run it.
### tensorflow-gpu==1.15.2
from nets.yolo4_tiny import yolo_body
from keras.layers import Input
image_input = Input(shape=(416, 416, 3))
model = yolo_body(image_input, 3, 20)
model.summary()
json_string = model.to_json()
open('yolov4_tiny_voc.json', 'w').write(json_string)
from pinto_model_zoo.
@PINTO0309 Thanks!
RaspberryPi4 + Ubuntu 19.10 aarch64 + ncnn + CPU only + 4 threads + YoloV4-tiny int8 416x416 326ms/pred
So int8 isn't faster on RPi4 + NCNN.
We should try to implement yolov4-tiny with Depthwise/Grouped convolutions.
RaspberryPi4 + CPU only + INT8 + Tensorflow Lite (4 threads) + 416x416 with 243ms/inference Performance.
RaspberryPi4 + Ubuntu 19.10 aarch64 + ncnn + CPU only + 4 threads + YoloV4-tiny 416x416 300ms/pred
So TensorFlow-Lite is 1.25x faster than NCNN.
from pinto_model_zoo.
Thanks !
As far as I know, the efficiency of ncnn int8 implementation is very poor, and it is normal that the speed is even slower than ncnn fp32.
I am currently working on fp16 and gpu acceleration, and I have no plan to optimize the efficiency of int8 in the coming weeks.
Maybe one day, I can't stand the speed of int8 anymore, I will try optimizing it 😃
from pinto_model_zoo.
I still think I'm transcribing the weights the wrong way. It's 1AM in Japan, so I'll try again tomorrow. 😄
from pinto_model_zoo.
It was very hard work, but it looks like I was able to refurbish openvino2tensorflow to generate the EdgeTPU model of YOLOv4-tiny. I found that there is a bug regarding the Resize OP conversion in either edgetpu_compiler or TFLiteConverter.
- EdgeTPU, YOLOv4-tiny, INT8
https://drive.google.com/file/d/1NsJlF-MVxBw7JCw60K1uFLTAlwdv-C0p/view?usp=sharing
from pinto_model_zoo.
@AlexeyAB Thank you for your reply.
Do you know if there is a plan to fix this?
No. I have posted similar issues, but so far I haven't received a definitive answer.
I don't know if Keras' implementation of YoloV4-tiny correctly replicates the original implementation, but I sympathize with you. I'm going to try to build OpenCV / NCNN myself for the first time in a long time. And I'm going to try it in Pi4.
from pinto_model_zoo.
@AlexeyAB
RaspberryPi4 + Ubuntu 19.10 aarch64 + ncnn + CPU only + 4 threads + YoloV4-tiny 416x416 300ms/pred
from pinto_model_zoo.
RaspberryPi4 (2.0GHz overclock) + Ubuntu 20.04 aarch64 + CPU only + INT8 + Tensorflow Lite (4 threads) + 416x416 with 224ms/inference Performance.
from pinto_model_zoo.
@AlexeyAB
I've created a script that automatically converts NCHW to NHWC. I will be adding more layers of support gradually.
https://github.com/PINTO0309/openvino2tensorflow.git
from pinto_model_zoo.
@AlexeyAB
Yes. It's hard to understand without reading all the logic, but all the Weights are converted to NHWC at the time of setting to Bias and Kernel for most of the layers. I'm just converting the weight to Const or np.ndarray temporarily and keeping it in dict.
from pinto_model_zoo.
Is keras.layers.Conv2D(filters=out_shape, kernel_size=3, data_format=None, groups=groups)
with groups > 1 suitable for this? https://www.tensorflow.org/api_docs/python/tf/keras/layers/Conv2D
Just maybe there is different layout [ky][kx][c][n]
or [ky][kx][n][c]
or something else.
from pinto_model_zoo.
I made such conversion of pt-weights to tflite-weights for EfficientNet-Lite3 successfully, and TFlite model works well: https://colab.research.google.com/gist/AlexeyAB/cc05f2690c3707d5e0f66d1e749b5546/weights_torch_to_tf_effnet_lite0.ipynb#scrollTo=WFnIID6iBlsq
I only converted weights, but the structure is taken from the official repository, there is such ReLU6 implementation tf.nn.relu6
: https://github.com/tensorflow/tpu/blob/f4ae890d33fc8d7abfa784622b505dc027a92ae7/models/official/efficientnet/lite/efficientnet_lite_builder.py#L71
The same as in your repo tf.nn.relu6
: https://github.com/PINTO0309/openvino2tensorflow/blob/5c85e312b105c5e7623f5e4d941334329e97e1da/openvino2tensorflow.py#L224
from pinto_model_zoo.
- for EfficientNet-lite: there is
groups == input_channels
, then try to usetf.keras.layers.DepthwiseConv2D
(you shouldn't specify number of groups there) https://keras.io/api/layers/convolution_layers/depthwise_convolution2d/- perhaps there will be layout
[ky][kx][c][n]
- perhaps there will be layout
- for ResNext (as in Midas): there is
groups != input_channels
, then try to use suchGroupedConv2D
implementation, just duplicate the same kernel_size several times in list: https://github.com/tensorflow/tpu/blob/2df828c20a2e9ac66690ff52768df68eab416265/models/official/mnasnet/mixnet/custom_layers.py#L24-L77
For example, forkernel_size=3x3
andgroups=4
use:
GroupedConv2D(
filters=out_filters,
kernel_size=[(3,3), (3,3), (3,3), (3,3)],
strides=[1, 1],
padding='same',
use_bias=False,
use_keras=True)
perhaps there will be layout [ky][kx][n][c]
from pinto_model_zoo.
@AlexeyAB
Btw, I successfully completed the conversion process to various frameworks two months ago.
- 016_EfficientNet-lite
https://github.com/PINTO0309/PINTO_model_zoo/tree/master/016_EfficientNet-lite
I did a crazy implementation, but the conversion to tflite appears to have succeeded. I have not checked the operation. What are the benefits of successfully completing this conversion process?
- tf_efficientnet_lite3_256x256_float32.tflite
https://drive.google.com/file/d/1qA9LdunxkHVnifC0dn7Upj02CckShF7t/view?usp=sharing
from pinto_model_zoo.
Great! It seems your model works well. I compare 3 models EfficientNet-Lite3 with the same dog.jpg
image: https://colab.research.google.com/gist/AlexeyAB/e81a12509f1043832fe096f65b7d5ea8/effnetlite3_256x256.ipynb
-
256x256: your tf_efficientnet_lite3_256x256_float32.tflite https://drive.google.com/file/d/1qA9LdunxkHVnifC0dn7Upj02CckShF7t/view?usp=sharing
-
280x280: TensorFlow-Hub https://tfhub.dev/tensorflow/lite-model/efficientnet/lite3/fp32/2
-
256x256: Pytorch: https://github.com/rwightman/gen-efficientnet-pytorch
I added Softmax at the end of 1 and 3, because 2 uses Softmax.
There are some differences here, possibly due to different normalization and different network resolutions:
1. TF-cst = [[839 266 912 505 891]]
2. TF-Hub = [[839 277 995 912 298]]
3. PT-hub = [[266 839 376 958 850]]
from pinto_model_zoo.
@PINTO0309 Sorry, it seems there is no error, my mistake ) Great work!
from pinto_model_zoo.
@AlexeyAB
It's not perfect, but I've written a workflow for converting PyTorch(NCHW) to TensorFlow(NHWC) in an article.
[English] Converting PyTorch, ONNX, Caffe, and OpenVINO (NCHW) models to Tensorflow / TensorflowLite (NHWC) in a snap - Qiita
Unfortunately, there is still a bug in the Reshape operation of the 5D tensor that causes YoloV4 and ShuffuleNet conversions to fail.
from pinto_model_zoo.
Is it because EdgeTPU doesn't support PReLU?
Yes. The PReLU was not present in the supported OPs listed at the following URL.
https://coral.ai/docs/edgetpu/models-intro/#supported-operations
The Transpose at the end was in the way, so I edited OpenVINO's .xml to remove it and then converted it to .tflite. It looks structurally sound, but I'm not sure if it works correctly.
- yolov4x-mish_float32.tflite
https://drive.google.com/file/d/1yQeizqDeNVTFJw1TqCH_vDcR5q96Xrfi/view?usp=sharing
from pinto_model_zoo.
How did you solve or avoid it?
I used the TensorFlow v2.x or later converters to pass the full-integer quantization model equivalent to resize_nearest_neighbor
or resize_bilinear
upsampling to edgetpu_compiler, and the I noticed that resize op is not properly converted to op for edgetpu. The problem was caused by the half_pixel_centers
of resize op being true
when doing full integer quantization.
google-coral/edgetpu#187
So I combined Lamda OP and tf.compat.v1.image.resize_bilinear
or tf.compat.v1.image.resize_nearest_neighbor
to make the half_pixel_centers
I tried how to force it to be set to False
. Below are the changes I made to openvino2tensorflow.
https://github.com/PINTO0309/openvino2tensorflow/blob/59796ce2160be62eb4da9a7200d45e0fd6c32f07/openvino2tensorflow/openvino2tensorflow.py#L565-L610
I have been playing with converting models that are committed to various repositories, so in this case I converted the models in the following repositories. keras -> openvino -> openvino2tensorflow -> EdgeTPU
https://github.com/bubbliiiing/yolov4-tiny-keras.git
The work I carry out is always fickle.
from pinto_model_zoo.
Just info for someone that does not like openvino path:
- there is also a path through keras (onnx2keras with change ordering parameter)
- you may need to tweak onnx2keras to get to to work but I managed to get tfjs model (without transpose mess) that takes 20ms instead of 68ms which is very nice :)
from pinto_model_zoo.
@PINTO0309 Thank you for your huge work as a hobby!
RaspberryPi4 + CPU only + INT8 + Tensorflow Lite (4 threads) + 416x416 with 243ms/inference Performance.
It seems it doesn't work fast on RPi4.
-
Did you try to test
yolov4-tiny + RPi4
by using new release OpenCV 4.4.0 with (DLIE-backend OpenVINO)? -
Or did you try to test
yolov4-tiny + RPi4
by using Tencent/NCNN? It seems yolov4-tiny achieves 20 FPS on ARM CPU Kirin 990 AlexeyAB/darknet#6091 (comment)
Unfortunately, the conversion to models for Coral TPU-Edge was not successful due to a bug in the Tensorflow Lite Converter.
Do you know if there is a plan to fix this?
Yes. I've described how to do the conversion in some of my blog posts below.
[English ver.] [Tensorflow Lite] Various Neural Network Model quantization methods for Tensorflow Lite (Weight Quantization, Integer Quantization, Full Integer Quantization, Float16 Quantization, EdgeTPU). As of May 05, 2020.
Alternatively, you can find a repository of perfect tutorials below.
https://github.com/leimao/Frozen_Graph_TensorFlow
Thanks, it helps a lot.
from pinto_model_zoo.
@PINTO0309 Thanks!
RaspberryPi4 + Ubuntu 19.10 aarch64 + ncnn + CPU only + 4 threads + YoloV4-tiny 416x416 300ms/pred
It seems yolov4-tiny speed is the same as mobilenet_yolo on RPi4.
Can you try to quantize yolov4-tiny to int8 and test it on RPi4? https://github.com/Tencent/ncnn/tree/master/tools/quantize#user-guide
If it will not help a lot, it seems we should try to implement yolov4-tiny with Depthwise/Grouped-convolution.
from pinto_model_zoo.
@AlexeyAB
After optimization and INT8 quantization, performance was mysteriously degraded.
RaspberryPi4 + Ubuntu 19.10 aarch64 + ncnn + CPU only + 4 threads + YoloV4-tiny int8 416x416 326ms/pred
- INT8 .bin, .param, .table
https://drive.google.com/drive/folders/1c3vB_lG7hxw2eqqhi5wVZh0yiKxjlEmP?usp=sharing
from pinto_model_zoo.
@nihui
Yes, GPU optimization is more important, especially on smartphones.
Are you using Vulkan or self-written functions for int8 inference?
from pinto_model_zoo.
@PINTO0309
Great!
So now we can convert any model:
PyTorch (NCHW) -> ONNX (NCHW) -> OpenVINO (NCHW) -> TF(pb) (NHWC) -> (NHWC) TFLite/TFJS/TF-TRT ... -> CoreML (NHWC)
- Does it convert weights too? I see that
binWeight
is used only for Const-layer: https://github.com/PINTO0309/openvino2tensorflow/blob/main/openvino2tensorflow.py#L134
from pinto_model_zoo.
@PINTO0309
It can be very useful!
-
Did you try to convert models like this? https://github.com/openvinotoolkit/open_model_zoo/tree/master/models/public/midasnet
-
Or default EfficientNet-Lite classification model?
import torch
model = torch.hub.load(
"rwightman/gen-efficientnet-pytorch",
"tf_efficientnet_lite3",
pretrained=True,
exportable=True
)
rand_example = torch.rand(1, 3, 256, 256)
output1 = model(rand_example)
traced_model = torch.jit.trace(model, rand_example)
scripted_model = torch.jit.script(model)
torch.onnx.export(model, rand_example, 'model.onnx', opset_version=10)
When I tried to do such conversion of tf_efficientnet_lite3
model PT->ONNX->TF->TFlite by using https://github.com/onnx/onnx-tensorflow then it isn't optimal and it can be run only on Mobile-CPU, but not on Mobile-GPU/NPU: onnx/onnx-tensorflow#782
from pinto_model_zoo.
@AlexeyAB
Thank you for providing useful information. I was just about to attempt the EfficientNet-B0-PyTorch conversion. However, I know that there is still a bug in the OP's conversion to manipulate the axis that prevents the conversion from finishing correctly.
https://github.com/openvinotoolkit/open_model_zoo/tree/master/models/public/efficientnet-b0-pytorch
I'm debugging a few things, so please be patient for a moment.
from pinto_model_zoo.
The midasnet groupcovolusion will probably need to be split with tf.keras.layers.SeparableConv2D or tf.nn.separable_conv2d. It looks like it needs a bit of a tricky implementation.
from pinto_model_zoo.
Oh... I'll try it when I get home!😄
from pinto_model_zoo.
@AlexeyAB
As it turns out, it worked. But, unfortunately, the protocol buffer size limit was exceeded and the timing of saving the model resulted in an error. The size of Midasnet seems to be too large for my inefficient conversion program. 😖
Convert Log
Layer (type) Output Shape Param # Connected to
==================================================================================================
image (InputLayer) [(1, 384, 384, 3)] 0
__________________________________________________________________________________________________
tf_op_layer_Mul (TensorFlowOpLa [(1, 384, 384, 3)] 0 image[0][0]
__________________________________________________________________________________________________
tf_op_layer_Add (TensorFlowOpLa [(1, 384, 384, 3)] 0 tf_op_layer_Mul[0][0]
__________________________________________________________________________________________________
conv2d (Conv2D) (1, 192, 192, 64) 9408 tf_op_layer_Add[0][0]
__________________________________________________________________________________________________
tf_op_layer_Add_1 (TensorFlowOp [(1, 192, 192, 64)] 0 conv2d[0][0]
__________________________________________________________________________________________________
re_lu (ReLU) (1, 192, 192, 64) 0 tf_op_layer_Add_1[0][0]
__________________________________________________________________________________________________
tf_op_layer_MaxPool (TensorFlow [(1, 96, 96, 64)] 0 re_lu[0][0]
__________________________________________________________________________________________________
conv2d_1 (Conv2D) (1, 96, 96, 256) 16384 tf_op_layer_MaxPool[0][0]
__________________________________________________________________________________________________
tf_op_layer_Add_2 (TensorFlowOp [(1, 96, 96, 256)] 0 conv2d_1[0][0]
__________________________________________________________________________________________________
re_lu_1 (ReLU) (1, 96, 96, 256) 0 tf_op_layer_Add_2[0][0]
__________________________________________________________________________________________________
conv2d_2 (Conv2D) (1, 96, 96, 256) 18432 re_lu_1[0][0]
__________________________________________________________________________________________________
tf_op_layer_Add_3 (TensorFlowOp [(1, 96, 96, 256)] 0 conv2d_2[0][0]
__________________________________________________________________________________________________
re_lu_2 (ReLU) (1, 96, 96, 256) 0 tf_op_layer_Add_3[0][0]
__________________________________________________________________________________________________
conv2d_3 (Conv2D) (1, 96, 96, 256) 65536 re_lu_2[0][0]
__________________________________________________________________________________________________
conv2d_4 (Conv2D) (1, 96, 96, 256) 16384 tf_op_layer_MaxPool[0][0]
__________________________________________________________________________________________________
tf_op_layer_Add_4 (TensorFlowOp [(1, 96, 96, 256)] 0 conv2d_3[0][0]
__________________________________________________________________________________________________
tf_op_layer_Add_5 (TensorFlowOp [(1, 96, 96, 256)] 0 conv2d_4[0][0]
__________________________________________________________________________________________________
add (Add) (1, 96, 96, 256) 0 tf_op_layer_Add_4[0][0]
tf_op_layer_Add_5[0][0]
__________________________________________________________________________________________________
re_lu_3 (ReLU) (1, 96, 96, 256) 0 add[0][0]
__________________________________________________________________________________________________
conv2d_5 (Conv2D) (1, 96, 96, 256) 65536 re_lu_3[0][0]
__________________________________________________________________________________________________
tf_op_layer_Add_6 (TensorFlowOp [(1, 96, 96, 256)] 0 conv2d_5[0][0]
__________________________________________________________________________________________________
re_lu_4 (ReLU) (1, 96, 96, 256) 0 tf_op_layer_Add_6[0][0]
__________________________________________________________________________________________________
conv2d_6 (Conv2D) (1, 96, 96, 256) 18432 re_lu_4[0][0]
__________________________________________________________________________________________________
tf_op_layer_Add_7 (TensorFlowOp [(1, 96, 96, 256)] 0 conv2d_6[0][0]
__________________________________________________________________________________________________
re_lu_5 (ReLU) (1, 96, 96, 256) 0 tf_op_layer_Add_7[0][0]
__________________________________________________________________________________________________
conv2d_7 (Conv2D) (1, 96, 96, 256) 65536 re_lu_5[0][0]
__________________________________________________________________________________________________
tf_op_layer_Add_8 (TensorFlowOp [(1, 96, 96, 256)] 0 conv2d_7[0][0]
__________________________________________________________________________________________________
add_1 (Add) (1, 96, 96, 256) 0 tf_op_layer_Add_8[0][0]
re_lu_3[0][0]
__________________________________________________________________________________________________
re_lu_6 (ReLU) (1, 96, 96, 256) 0 add_1[0][0]
__________________________________________________________________________________________________
conv2d_8 (Conv2D) (1, 96, 96, 256) 65536 re_lu_6[0][0]
__________________________________________________________________________________________________
tf_op_layer_Add_9 (TensorFlowOp [(1, 96, 96, 256)] 0 conv2d_8[0][0]
__________________________________________________________________________________________________
re_lu_7 (ReLU) (1, 96, 96, 256) 0 tf_op_layer_Add_9[0][0]
__________________________________________________________________________________________________
conv2d_9 (Conv2D) (1, 96, 96, 256) 18432 re_lu_7[0][0]
__________________________________________________________________________________________________
tf_op_layer_Add_10 (TensorFlowO [(1, 96, 96, 256)] 0 conv2d_9[0][0]
__________________________________________________________________________________________________
re_lu_8 (ReLU) (1, 96, 96, 256) 0 tf_op_layer_Add_10[0][0]
__________________________________________________________________________________________________
conv2d_10 (Conv2D) (1, 96, 96, 256) 65536 re_lu_8[0][0]
__________________________________________________________________________________________________
tf_op_layer_Add_11 (TensorFlowO [(1, 96, 96, 256)] 0 conv2d_10[0][0]
__________________________________________________________________________________________________
add_2 (Add) (1, 96, 96, 256) 0 tf_op_layer_Add_11[0][0]
re_lu_6[0][0]
__________________________________________________________________________________________________
re_lu_9 (ReLU) (1, 96, 96, 256) 0 add_2[0][0]
__________________________________________________________________________________________________
conv2d_11 (Conv2D) (1, 96, 96, 512) 131072 re_lu_9[0][0]
__________________________________________________________________________________________________
tf_op_layer_Add_12 (TensorFlowO [(1, 96, 96, 512)] 0 conv2d_11[0][0]
__________________________________________________________________________________________________
re_lu_10 (ReLU) (1, 96, 96, 512) 0 tf_op_layer_Add_12[0][0]
__________________________________________________________________________________________________
conv2d_12 (Conv2D) (1, 48, 48, 512) 73728 re_lu_10[0][0]
__________________________________________________________________________________________________
tf_op_layer_Add_13 (TensorFlowO [(1, 48, 48, 512)] 0 conv2d_12[0][0]
__________________________________________________________________________________________________
re_lu_11 (ReLU) (1, 48, 48, 512) 0 tf_op_layer_Add_13[0][0]
__________________________________________________________________________________________________
conv2d_13 (Conv2D) (1, 48, 48, 512) 262144 re_lu_11[0][0]
__________________________________________________________________________________________________
conv2d_14 (Conv2D) (1, 48, 48, 512) 131072 re_lu_9[0][0]
__________________________________________________________________________________________________
tf_op_layer_Add_14 (TensorFlowO [(1, 48, 48, 512)] 0 conv2d_13[0][0]
__________________________________________________________________________________________________
tf_op_layer_Add_15 (TensorFlowO [(1, 48, 48, 512)] 0 conv2d_14[0][0]
__________________________________________________________________________________________________
add_3 (Add) (1, 48, 48, 512) 0 tf_op_layer_Add_14[0][0]
tf_op_layer_Add_15[0][0]
__________________________________________________________________________________________________
re_lu_12 (ReLU) (1, 48, 48, 512) 0 add_3[0][0]
__________________________________________________________________________________________________
conv2d_15 (Conv2D) (1, 48, 48, 512) 262144 re_lu_12[0][0]
__________________________________________________________________________________________________
tf_op_layer_Add_16 (TensorFlowO [(1, 48, 48, 512)] 0 conv2d_15[0][0]
__________________________________________________________________________________________________
re_lu_13 (ReLU) (1, 48, 48, 512) 0 tf_op_layer_Add_16[0][0]
__________________________________________________________________________________________________
conv2d_16 (Conv2D) (1, 48, 48, 512) 73728 re_lu_13[0][0]
__________________________________________________________________________________________________
tf_op_layer_Add_17 (TensorFlowO [(1, 48, 48, 512)] 0 conv2d_16[0][0]
__________________________________________________________________________________________________
re_lu_14 (ReLU) (1, 48, 48, 512) 0 tf_op_layer_Add_17[0][0]
__________________________________________________________________________________________________
conv2d_17 (Conv2D) (1, 48, 48, 512) 262144 re_lu_14[0][0]
__________________________________________________________________________________________________
tf_op_layer_Add_18 (TensorFlowO [(1, 48, 48, 512)] 0 conv2d_17[0][0]
__________________________________________________________________________________________________
add_4 (Add) (1, 48, 48, 512) 0 tf_op_layer_Add_18[0][0]
re_lu_12[0][0]
__________________________________________________________________________________________________
re_lu_15 (ReLU) (1, 48, 48, 512) 0 add_4[0][0]
__________________________________________________________________________________________________
conv2d_18 (Conv2D) (1, 48, 48, 512) 262144 re_lu_15[0][0]
__________________________________________________________________________________________________
tf_op_layer_Add_19 (TensorFlowO [(1, 48, 48, 512)] 0 conv2d_18[0][0]
__________________________________________________________________________________________________
re_lu_16 (ReLU) (1, 48, 48, 512) 0 tf_op_layer_Add_19[0][0]
__________________________________________________________________________________________________
conv2d_19 (Conv2D) (1, 48, 48, 512) 73728 re_lu_16[0][0]
__________________________________________________________________________________________________
tf_op_layer_Add_20 (TensorFlowO [(1, 48, 48, 512)] 0 conv2d_19[0][0]
__________________________________________________________________________________________________
re_lu_17 (ReLU) (1, 48, 48, 512) 0 tf_op_layer_Add_20[0][0]
__________________________________________________________________________________________________
conv2d_20 (Conv2D) (1, 48, 48, 512) 262144 re_lu_17[0][0]
__________________________________________________________________________________________________
tf_op_layer_Add_21 (TensorFlowO [(1, 48, 48, 512)] 0 conv2d_20[0][0]
__________________________________________________________________________________________________
add_5 (Add) (1, 48, 48, 512) 0 tf_op_layer_Add_21[0][0]
re_lu_15[0][0]
__________________________________________________________________________________________________
re_lu_18 (ReLU) (1, 48, 48, 512) 0 add_5[0][0]
__________________________________________________________________________________________________
conv2d_21 (Conv2D) (1, 48, 48, 512) 262144 re_lu_18[0][0]
__________________________________________________________________________________________________
tf_op_layer_Add_22 (TensorFlowO [(1, 48, 48, 512)] 0 conv2d_21[0][0]
__________________________________________________________________________________________________
re_lu_19 (ReLU) (1, 48, 48, 512) 0 tf_op_layer_Add_22[0][0]
__________________________________________________________________________________________________
conv2d_22 (Conv2D) (1, 48, 48, 512) 73728 re_lu_19[0][0]
__________________________________________________________________________________________________
tf_op_layer_Add_23 (TensorFlowO [(1, 48, 48, 512)] 0 conv2d_22[0][0]
__________________________________________________________________________________________________
re_lu_20 (ReLU) (1, 48, 48, 512) 0 tf_op_layer_Add_23[0][0]
__________________________________________________________________________________________________
conv2d_23 (Conv2D) (1, 48, 48, 512) 262144 re_lu_20[0][0]
__________________________________________________________________________________________________
tf_op_layer_Add_24 (TensorFlowO [(1, 48, 48, 512)] 0 conv2d_23[0][0]
__________________________________________________________________________________________________
add_6 (Add) (1, 48, 48, 512) 0 tf_op_layer_Add_24[0][0]
re_lu_18[0][0]
__________________________________________________________________________________________________
re_lu_21 (ReLU) (1, 48, 48, 512) 0 add_6[0][0]
__________________________________________________________________________________________________
conv2d_24 (Conv2D) (1, 48, 48, 1024) 524288 re_lu_21[0][0]
__________________________________________________________________________________________________
tf_op_layer_Add_25 (TensorFlowO [(1, 48, 48, 1024)] 0 conv2d_24[0][0]
__________________________________________________________________________________________________
re_lu_22 (ReLU) (1, 48, 48, 1024) 0 tf_op_layer_Add_25[0][0]
__________________________________________________________________________________________________
conv2d_25 (Conv2D) (1, 24, 24, 1024) 294912 re_lu_22[0][0]
__________________________________________________________________________________________________
tf_op_layer_Add_26 (TensorFlowO [(1, 24, 24, 1024)] 0 conv2d_25[0][0]
__________________________________________________________________________________________________
re_lu_23 (ReLU) (1, 24, 24, 1024) 0 tf_op_layer_Add_26[0][0]
__________________________________________________________________________________________________
conv2d_26 (Conv2D) (1, 24, 24, 1024) 1048576 re_lu_23[0][0]
__________________________________________________________________________________________________
conv2d_27 (Conv2D) (1, 24, 24, 1024) 524288 re_lu_21[0][0]
__________________________________________________________________________________________________
tf_op_layer_Add_27 (TensorFlowO [(1, 24, 24, 1024)] 0 conv2d_26[0][0]
__________________________________________________________________________________________________
tf_op_layer_Add_28 (TensorFlowO [(1, 24, 24, 1024)] 0 conv2d_27[0][0]
__________________________________________________________________________________________________
add_7 (Add) (1, 24, 24, 1024) 0 tf_op_layer_Add_27[0][0]
tf_op_layer_Add_28[0][0]
__________________________________________________________________________________________________
re_lu_24 (ReLU) (1, 24, 24, 1024) 0 add_7[0][0]
__________________________________________________________________________________________________
conv2d_28 (Conv2D) (1, 24, 24, 1024) 1048576 re_lu_24[0][0]
__________________________________________________________________________________________________
tf_op_layer_Add_29 (TensorFlowO [(1, 24, 24, 1024)] 0 conv2d_28[0][0]
__________________________________________________________________________________________________
re_lu_25 (ReLU) (1, 24, 24, 1024) 0 tf_op_layer_Add_29[0][0]
__________________________________________________________________________________________________
conv2d_29 (Conv2D) (1, 24, 24, 1024) 294912 re_lu_25[0][0]
__________________________________________________________________________________________________
tf_op_layer_Add_30 (TensorFlowO [(1, 24, 24, 1024)] 0 conv2d_29[0][0]
__________________________________________________________________________________________________
re_lu_26 (ReLU) (1, 24, 24, 1024) 0 tf_op_layer_Add_30[0][0]
__________________________________________________________________________________________________
conv2d_30 (Conv2D) (1, 24, 24, 1024) 1048576 re_lu_26[0][0]
__________________________________________________________________________________________________
tf_op_layer_Add_31 (TensorFlowO [(1, 24, 24, 1024)] 0 conv2d_30[0][0]
__________________________________________________________________________________________________
add_8 (Add) (1, 24, 24, 1024) 0 tf_op_layer_Add_31[0][0]
re_lu_24[0][0]
__________________________________________________________________________________________________
re_lu_27 (ReLU) (1, 24, 24, 1024) 0 add_8[0][0]
__________________________________________________________________________________________________
conv2d_31 (Conv2D) (1, 24, 24, 1024) 1048576 re_lu_27[0][0]
__________________________________________________________________________________________________
tf_op_layer_Add_32 (TensorFlowO [(1, 24, 24, 1024)] 0 conv2d_31[0][0]
__________________________________________________________________________________________________
re_lu_28 (ReLU) (1, 24, 24, 1024) 0 tf_op_layer_Add_32[0][0]
__________________________________________________________________________________________________
conv2d_32 (Conv2D) (1, 24, 24, 1024) 294912 re_lu_28[0][0]
__________________________________________________________________________________________________
tf_op_layer_Add_33 (TensorFlowO [(1, 24, 24, 1024)] 0 conv2d_32[0][0]
__________________________________________________________________________________________________
re_lu_29 (ReLU) (1, 24, 24, 1024) 0 tf_op_layer_Add_33[0][0]
__________________________________________________________________________________________________
conv2d_33 (Conv2D) (1, 24, 24, 1024) 1048576 re_lu_29[0][0]
__________________________________________________________________________________________________
tf_op_layer_Add_34 (TensorFlowO [(1, 24, 24, 1024)] 0 conv2d_33[0][0]
__________________________________________________________________________________________________
add_9 (Add) (1, 24, 24, 1024) 0 tf_op_layer_Add_34[0][0]
re_lu_27[0][0]
__________________________________________________________________________________________________
re_lu_30 (ReLU) (1, 24, 24, 1024) 0 add_9[0][0]
__________________________________________________________________________________________________
conv2d_34 (Conv2D) (1, 24, 24, 1024) 1048576 re_lu_30[0][0]
__________________________________________________________________________________________________
tf_op_layer_Add_35 (TensorFlowO [(1, 24, 24, 1024)] 0 conv2d_34[0][0]
__________________________________________________________________________________________________
re_lu_31 (ReLU) (1, 24, 24, 1024) 0 tf_op_layer_Add_35[0][0]
__________________________________________________________________________________________________
conv2d_35 (Conv2D) (1, 24, 24, 1024) 294912 re_lu_31[0][0]
__________________________________________________________________________________________________
tf_op_layer_Add_36 (TensorFlowO [(1, 24, 24, 1024)] 0 conv2d_35[0][0]
__________________________________________________________________________________________________
re_lu_32 (ReLU) (1, 24, 24, 1024) 0 tf_op_layer_Add_36[0][0]
__________________________________________________________________________________________________
conv2d_36 (Conv2D) (1, 24, 24, 1024) 1048576 re_lu_32[0][0]
__________________________________________________________________________________________________
tf_op_layer_Add_37 (TensorFlowO [(1, 24, 24, 1024)] 0 conv2d_36[0][0]
__________________________________________________________________________________________________
add_10 (Add) (1, 24, 24, 1024) 0 tf_op_layer_Add_37[0][0]
re_lu_30[0][0]
__________________________________________________________________________________________________
re_lu_33 (ReLU) (1, 24, 24, 1024) 0 add_10[0][0]
__________________________________________________________________________________________________
conv2d_37 (Conv2D) (1, 24, 24, 1024) 1048576 re_lu_33[0][0]
__________________________________________________________________________________________________
tf_op_layer_Add_38 (TensorFlowO [(1, 24, 24, 1024)] 0 conv2d_37[0][0]
__________________________________________________________________________________________________
re_lu_34 (ReLU) (1, 24, 24, 1024) 0 tf_op_layer_Add_38[0][0]
__________________________________________________________________________________________________
conv2d_38 (Conv2D) (1, 24, 24, 1024) 294912 re_lu_34[0][0]
__________________________________________________________________________________________________
tf_op_layer_Add_39 (TensorFlowO [(1, 24, 24, 1024)] 0 conv2d_38[0][0]
__________________________________________________________________________________________________
re_lu_35 (ReLU) (1, 24, 24, 1024) 0 tf_op_layer_Add_39[0][0]
__________________________________________________________________________________________________
conv2d_39 (Conv2D) (1, 24, 24, 1024) 1048576 re_lu_35[0][0]
__________________________________________________________________________________________________
tf_op_layer_Add_40 (TensorFlowO [(1, 24, 24, 1024)] 0 conv2d_39[0][0]
__________________________________________________________________________________________________
add_11 (Add) (1, 24, 24, 1024) 0 tf_op_layer_Add_40[0][0]
re_lu_33[0][0]
__________________________________________________________________________________________________
re_lu_36 (ReLU) (1, 24, 24, 1024) 0 add_11[0][0]
__________________________________________________________________________________________________
conv2d_40 (Conv2D) (1, 24, 24, 1024) 1048576 re_lu_36[0][0]
__________________________________________________________________________________________________
tf_op_layer_Add_41 (TensorFlowO [(1, 24, 24, 1024)] 0 conv2d_40[0][0]
__________________________________________________________________________________________________
re_lu_37 (ReLU) (1, 24, 24, 1024) 0 tf_op_layer_Add_41[0][0]
__________________________________________________________________________________________________
conv2d_41 (Conv2D) (1, 24, 24, 1024) 294912 re_lu_37[0][0]
__________________________________________________________________________________________________
tf_op_layer_Add_42 (TensorFlowO [(1, 24, 24, 1024)] 0 conv2d_41[0][0]
__________________________________________________________________________________________________
re_lu_38 (ReLU) (1, 24, 24, 1024) 0 tf_op_layer_Add_42[0][0]
__________________________________________________________________________________________________
conv2d_42 (Conv2D) (1, 24, 24, 1024) 1048576 re_lu_38[0][0]
__________________________________________________________________________________________________
tf_op_layer_Add_43 (TensorFlowO [(1, 24, 24, 1024)] 0 conv2d_42[0][0]
__________________________________________________________________________________________________
add_12 (Add) (1, 24, 24, 1024) 0 tf_op_layer_Add_43[0][0]
re_lu_36[0][0]
__________________________________________________________________________________________________
re_lu_39 (ReLU) (1, 24, 24, 1024) 0 add_12[0][0]
__________________________________________________________________________________________________
conv2d_43 (Conv2D) (1, 24, 24, 1024) 1048576 re_lu_39[0][0]
__________________________________________________________________________________________________
tf_op_layer_Add_44 (TensorFlowO [(1, 24, 24, 1024)] 0 conv2d_43[0][0]
__________________________________________________________________________________________________
re_lu_40 (ReLU) (1, 24, 24, 1024) 0 tf_op_layer_Add_44[0][0]
__________________________________________________________________________________________________
conv2d_44 (Conv2D) (1, 24, 24, 1024) 294912 re_lu_40[0][0]
__________________________________________________________________________________________________
tf_op_layer_Add_45 (TensorFlowO [(1, 24, 24, 1024)] 0 conv2d_44[0][0]
__________________________________________________________________________________________________
re_lu_41 (ReLU) (1, 24, 24, 1024) 0 tf_op_layer_Add_45[0][0]
__________________________________________________________________________________________________
conv2d_45 (Conv2D) (1, 24, 24, 1024) 1048576 re_lu_41[0][0]
__________________________________________________________________________________________________
tf_op_layer_Add_46 (TensorFlowO [(1, 24, 24, 1024)] 0 conv2d_45[0][0]
__________________________________________________________________________________________________
add_13 (Add) (1, 24, 24, 1024) 0 tf_op_layer_Add_46[0][0]
re_lu_39[0][0]
__________________________________________________________________________________________________
re_lu_42 (ReLU) (1, 24, 24, 1024) 0 add_13[0][0]
__________________________________________________________________________________________________
conv2d_46 (Conv2D) (1, 24, 24, 1024) 1048576 re_lu_42[0][0]
__________________________________________________________________________________________________
tf_op_layer_Add_47 (TensorFlowO [(1, 24, 24, 1024)] 0 conv2d_46[0][0]
__________________________________________________________________________________________________
re_lu_43 (ReLU) (1, 24, 24, 1024) 0 tf_op_layer_Add_47[0][0]
__________________________________________________________________________________________________
conv2d_47 (Conv2D) (1, 24, 24, 1024) 294912 re_lu_43[0][0]
__________________________________________________________________________________________________
tf_op_layer_Add_48 (TensorFlowO [(1, 24, 24, 1024)] 0 conv2d_47[0][0]
__________________________________________________________________________________________________
re_lu_44 (ReLU) (1, 24, 24, 1024) 0 tf_op_layer_Add_48[0][0]
__________________________________________________________________________________________________
conv2d_48 (Conv2D) (1, 24, 24, 1024) 1048576 re_lu_44[0][0]
__________________________________________________________________________________________________
tf_op_layer_Add_49 (TensorFlowO [(1, 24, 24, 1024)] 0 conv2d_48[0][0]
__________________________________________________________________________________________________
add_14 (Add) (1, 24, 24, 1024) 0 tf_op_layer_Add_49[0][0]
re_lu_42[0][0]
__________________________________________________________________________________________________
re_lu_45 (ReLU) (1, 24, 24, 1024) 0 add_14[0][0]
__________________________________________________________________________________________________
conv2d_49 (Conv2D) (1, 24, 24, 1024) 1048576 re_lu_45[0][0]
__________________________________________________________________________________________________
tf_op_layer_Add_50 (TensorFlowO [(1, 24, 24, 1024)] 0 conv2d_49[0][0]
__________________________________________________________________________________________________
re_lu_46 (ReLU) (1, 24, 24, 1024) 0 tf_op_layer_Add_50[0][0]
__________________________________________________________________________________________________
conv2d_50 (Conv2D) (1, 24, 24, 1024) 294912 re_lu_46[0][0]
__________________________________________________________________________________________________
tf_op_layer_Add_51 (TensorFlowO [(1, 24, 24, 1024)] 0 conv2d_50[0][0]
__________________________________________________________________________________________________
re_lu_47 (ReLU) (1, 24, 24, 1024) 0 tf_op_layer_Add_51[0][0]
__________________________________________________________________________________________________
conv2d_51 (Conv2D) (1, 24, 24, 1024) 1048576 re_lu_47[0][0]
__________________________________________________________________________________________________
tf_op_layer_Add_52 (TensorFlowO [(1, 24, 24, 1024)] 0 conv2d_51[0][0]
__________________________________________________________________________________________________
add_15 (Add) (1, 24, 24, 1024) 0 tf_op_layer_Add_52[0][0]
re_lu_45[0][0]
__________________________________________________________________________________________________
re_lu_48 (ReLU) (1, 24, 24, 1024) 0 add_15[0][0]
__________________________________________________________________________________________________
conv2d_52 (Conv2D) (1, 24, 24, 1024) 1048576 re_lu_48[0][0]
__________________________________________________________________________________________________
tf_op_layer_Add_53 (TensorFlowO [(1, 24, 24, 1024)] 0 conv2d_52[0][0]
__________________________________________________________________________________________________
re_lu_49 (ReLU) (1, 24, 24, 1024) 0 tf_op_layer_Add_53[0][0]
__________________________________________________________________________________________________
conv2d_53 (Conv2D) (1, 24, 24, 1024) 294912 re_lu_49[0][0]
__________________________________________________________________________________________________
tf_op_layer_Add_54 (TensorFlowO [(1, 24, 24, 1024)] 0 conv2d_53[0][0]
__________________________________________________________________________________________________
re_lu_50 (ReLU) (1, 24, 24, 1024) 0 tf_op_layer_Add_54[0][0]
__________________________________________________________________________________________________
conv2d_54 (Conv2D) (1, 24, 24, 1024) 1048576 re_lu_50[0][0]
__________________________________________________________________________________________________
tf_op_layer_Add_55 (TensorFlowO [(1, 24, 24, 1024)] 0 conv2d_54[0][0]
__________________________________________________________________________________________________
add_16 (Add) (1, 24, 24, 1024) 0 tf_op_layer_Add_55[0][0]
re_lu_48[0][0]
__________________________________________________________________________________________________
re_lu_51 (ReLU) (1, 24, 24, 1024) 0 add_16[0][0]
__________________________________________________________________________________________________
conv2d_55 (Conv2D) (1, 24, 24, 1024) 1048576 re_lu_51[0][0]
__________________________________________________________________________________________________
tf_op_layer_Add_56 (TensorFlowO [(1, 24, 24, 1024)] 0 conv2d_55[0][0]
__________________________________________________________________________________________________
re_lu_52 (ReLU) (1, 24, 24, 1024) 0 tf_op_layer_Add_56[0][0]
__________________________________________________________________________________________________
conv2d_56 (Conv2D) (1, 24, 24, 1024) 294912 re_lu_52[0][0]
__________________________________________________________________________________________________
tf_op_layer_Add_57 (TensorFlowO [(1, 24, 24, 1024)] 0 conv2d_56[0][0]
__________________________________________________________________________________________________
re_lu_53 (ReLU) (1, 24, 24, 1024) 0 tf_op_layer_Add_57[0][0]
__________________________________________________________________________________________________
conv2d_57 (Conv2D) (1, 24, 24, 1024) 1048576 re_lu_53[0][0]
__________________________________________________________________________________________________
tf_op_layer_Add_58 (TensorFlowO [(1, 24, 24, 1024)] 0 conv2d_57[0][0]
__________________________________________________________________________________________________
add_17 (Add) (1, 24, 24, 1024) 0 tf_op_layer_Add_58[0][0]
re_lu_51[0][0]
__________________________________________________________________________________________________
re_lu_54 (ReLU) (1, 24, 24, 1024) 0 add_17[0][0]
__________________________________________________________________________________________________
conv2d_58 (Conv2D) (1, 24, 24, 1024) 1048576 re_lu_54[0][0]
__________________________________________________________________________________________________
tf_op_layer_Add_59 (TensorFlowO [(1, 24, 24, 1024)] 0 conv2d_58[0][0]
__________________________________________________________________________________________________
re_lu_55 (ReLU) (1, 24, 24, 1024) 0 tf_op_layer_Add_59[0][0]
__________________________________________________________________________________________________
conv2d_59 (Conv2D) (1, 24, 24, 1024) 294912 re_lu_55[0][0]
__________________________________________________________________________________________________
tf_op_layer_Add_60 (TensorFlowO [(1, 24, 24, 1024)] 0 conv2d_59[0][0]
__________________________________________________________________________________________________
re_lu_56 (ReLU) (1, 24, 24, 1024) 0 tf_op_layer_Add_60[0][0]
__________________________________________________________________________________________________
conv2d_60 (Conv2D) (1, 24, 24, 1024) 1048576 re_lu_56[0][0]
__________________________________________________________________________________________________
tf_op_layer_Add_61 (TensorFlowO [(1, 24, 24, 1024)] 0 conv2d_60[0][0]
__________________________________________________________________________________________________
add_18 (Add) (1, 24, 24, 1024) 0 tf_op_layer_Add_61[0][0]
re_lu_54[0][0]
__________________________________________________________________________________________________
re_lu_57 (ReLU) (1, 24, 24, 1024) 0 add_18[0][0]
__________________________________________________________________________________________________
conv2d_61 (Conv2D) (1, 24, 24, 1024) 1048576 re_lu_57[0][0]
__________________________________________________________________________________________________
tf_op_layer_Add_62 (TensorFlowO [(1, 24, 24, 1024)] 0 conv2d_61[0][0]
__________________________________________________________________________________________________
re_lu_58 (ReLU) (1, 24, 24, 1024) 0 tf_op_layer_Add_62[0][0]
__________________________________________________________________________________________________
conv2d_62 (Conv2D) (1, 24, 24, 1024) 294912 re_lu_58[0][0]
__________________________________________________________________________________________________
tf_op_layer_Add_63 (TensorFlowO [(1, 24, 24, 1024)] 0 conv2d_62[0][0]
__________________________________________________________________________________________________
re_lu_59 (ReLU) (1, 24, 24, 1024) 0 tf_op_layer_Add_63[0][0]
__________________________________________________________________________________________________
conv2d_63 (Conv2D) (1, 24, 24, 1024) 1048576 re_lu_59[0][0]
__________________________________________________________________________________________________
tf_op_layer_Add_64 (TensorFlowO [(1, 24, 24, 1024)] 0 conv2d_63[0][0]
__________________________________________________________________________________________________
add_19 (Add) (1, 24, 24, 1024) 0 tf_op_layer_Add_64[0][0]
re_lu_57[0][0]
__________________________________________________________________________________________________
re_lu_60 (ReLU) (1, 24, 24, 1024) 0 add_19[0][0]
__________________________________________________________________________________________________
conv2d_64 (Conv2D) (1, 24, 24, 1024) 1048576 re_lu_60[0][0]
__________________________________________________________________________________________________
tf_op_layer_Add_65 (TensorFlowO [(1, 24, 24, 1024)] 0 conv2d_64[0][0]
__________________________________________________________________________________________________
re_lu_61 (ReLU) (1, 24, 24, 1024) 0 tf_op_layer_Add_65[0][0]
__________________________________________________________________________________________________
conv2d_65 (Conv2D) (1, 24, 24, 1024) 294912 re_lu_61[0][0]
__________________________________________________________________________________________________
tf_op_layer_Add_66 (TensorFlowO [(1, 24, 24, 1024)] 0 conv2d_65[0][0]
__________________________________________________________________________________________________
re_lu_62 (ReLU) (1, 24, 24, 1024) 0 tf_op_layer_Add_66[0][0]
__________________________________________________________________________________________________
conv2d_66 (Conv2D) (1, 24, 24, 1024) 1048576 re_lu_62[0][0]
__________________________________________________________________________________________________
tf_op_layer_Add_67 (TensorFlowO [(1, 24, 24, 1024)] 0 conv2d_66[0][0]
__________________________________________________________________________________________________
add_20 (Add) (1, 24, 24, 1024) 0 tf_op_layer_Add_67[0][0]
re_lu_60[0][0]
__________________________________________________________________________________________________
re_lu_63 (ReLU) (1, 24, 24, 1024) 0 add_20[0][0]
__________________________________________________________________________________________________
conv2d_67 (Conv2D) (1, 24, 24, 1024) 1048576 re_lu_63[0][0]
__________________________________________________________________________________________________
tf_op_layer_Add_68 (TensorFlowO [(1, 24, 24, 1024)] 0 conv2d_67[0][0]
__________________________________________________________________________________________________
re_lu_64 (ReLU) (1, 24, 24, 1024) 0 tf_op_layer_Add_68[0][0]
__________________________________________________________________________________________________
conv2d_68 (Conv2D) (1, 24, 24, 1024) 294912 re_lu_64[0][0]
__________________________________________________________________________________________________
tf_op_layer_Add_69 (TensorFlowO [(1, 24, 24, 1024)] 0 conv2d_68[0][0]
__________________________________________________________________________________________________
re_lu_65 (ReLU) (1, 24, 24, 1024) 0 tf_op_layer_Add_69[0][0]
__________________________________________________________________________________________________
conv2d_69 (Conv2D) (1, 24, 24, 1024) 1048576 re_lu_65[0][0]
__________________________________________________________________________________________________
tf_op_layer_Add_70 (TensorFlowO [(1, 24, 24, 1024)] 0 conv2d_69[0][0]
__________________________________________________________________________________________________
add_21 (Add) (1, 24, 24, 1024) 0 tf_op_layer_Add_70[0][0]
re_lu_63[0][0]
__________________________________________________________________________________________________
re_lu_66 (ReLU) (1, 24, 24, 1024) 0 add_21[0][0]
__________________________________________________________________________________________________
conv2d_70 (Conv2D) (1, 24, 24, 1024) 1048576 re_lu_66[0][0]
__________________________________________________________________________________________________
tf_op_layer_Add_71 (TensorFlowO [(1, 24, 24, 1024)] 0 conv2d_70[0][0]
__________________________________________________________________________________________________
re_lu_67 (ReLU) (1, 24, 24, 1024) 0 tf_op_layer_Add_71[0][0]
__________________________________________________________________________________________________
conv2d_71 (Conv2D) (1, 24, 24, 1024) 294912 re_lu_67[0][0]
__________________________________________________________________________________________________
tf_op_layer_Add_72 (TensorFlowO [(1, 24, 24, 1024)] 0 conv2d_71[0][0]
__________________________________________________________________________________________________
re_lu_68 (ReLU) (1, 24, 24, 1024) 0 tf_op_layer_Add_72[0][0]
__________________________________________________________________________________________________
conv2d_72 (Conv2D) (1, 24, 24, 1024) 1048576 re_lu_68[0][0]
__________________________________________________________________________________________________
tf_op_layer_Add_73 (TensorFlowO [(1, 24, 24, 1024)] 0 conv2d_72[0][0]
__________________________________________________________________________________________________
add_22 (Add) (1, 24, 24, 1024) 0 tf_op_layer_Add_73[0][0]
re_lu_66[0][0]
__________________________________________________________________________________________________
re_lu_69 (ReLU) (1, 24, 24, 1024) 0 add_22[0][0]
__________________________________________________________________________________________________
conv2d_73 (Conv2D) (1, 24, 24, 1024) 1048576 re_lu_69[0][0]
__________________________________________________________________________________________________
tf_op_layer_Add_74 (TensorFlowO [(1, 24, 24, 1024)] 0 conv2d_73[0][0]
__________________________________________________________________________________________________
re_lu_70 (ReLU) (1, 24, 24, 1024) 0 tf_op_layer_Add_74[0][0]
__________________________________________________________________________________________________
conv2d_74 (Conv2D) (1, 24, 24, 1024) 294912 re_lu_70[0][0]
__________________________________________________________________________________________________
tf_op_layer_Add_75 (TensorFlowO [(1, 24, 24, 1024)] 0 conv2d_74[0][0]
__________________________________________________________________________________________________
re_lu_71 (ReLU) (1, 24, 24, 1024) 0 tf_op_layer_Add_75[0][0]
__________________________________________________________________________________________________
conv2d_75 (Conv2D) (1, 24, 24, 1024) 1048576 re_lu_71[0][0]
__________________________________________________________________________________________________
tf_op_layer_Add_76 (TensorFlowO [(1, 24, 24, 1024)] 0 conv2d_75[0][0]
__________________________________________________________________________________________________
add_23 (Add) (1, 24, 24, 1024) 0 tf_op_layer_Add_76[0][0]
re_lu_69[0][0]
__________________________________________________________________________________________________
re_lu_72 (ReLU) (1, 24, 24, 1024) 0 add_23[0][0]
__________________________________________________________________________________________________
conv2d_76 (Conv2D) (1, 24, 24, 1024) 1048576 re_lu_72[0][0]
__________________________________________________________________________________________________
tf_op_layer_Add_77 (TensorFlowO [(1, 24, 24, 1024)] 0 conv2d_76[0][0]
__________________________________________________________________________________________________
re_lu_73 (ReLU) (1, 24, 24, 1024) 0 tf_op_layer_Add_77[0][0]
__________________________________________________________________________________________________
conv2d_77 (Conv2D) (1, 24, 24, 1024) 294912 re_lu_73[0][0]
__________________________________________________________________________________________________
tf_op_layer_Add_78 (TensorFlowO [(1, 24, 24, 1024)] 0 conv2d_77[0][0]
__________________________________________________________________________________________________
re_lu_74 (ReLU) (1, 24, 24, 1024) 0 tf_op_layer_Add_78[0][0]
__________________________________________________________________________________________________
conv2d_78 (Conv2D) (1, 24, 24, 1024) 1048576 re_lu_74[0][0]
__________________________________________________________________________________________________
tf_op_layer_Add_79 (TensorFlowO [(1, 24, 24, 1024)] 0 conv2d_78[0][0]
__________________________________________________________________________________________________
add_24 (Add) (1, 24, 24, 1024) 0 tf_op_layer_Add_79[0][0]
re_lu_72[0][0]
__________________________________________________________________________________________________
re_lu_75 (ReLU) (1, 24, 24, 1024) 0 add_24[0][0]
__________________________________________________________________________________________________
conv2d_79 (Conv2D) (1, 24, 24, 1024) 1048576 re_lu_75[0][0]
__________________________________________________________________________________________________
tf_op_layer_Add_80 (TensorFlowO [(1, 24, 24, 1024)] 0 conv2d_79[0][0]
__________________________________________________________________________________________________
re_lu_76 (ReLU) (1, 24, 24, 1024) 0 tf_op_layer_Add_80[0][0]
__________________________________________________________________________________________________
conv2d_80 (Conv2D) (1, 24, 24, 1024) 294912 re_lu_76[0][0]
__________________________________________________________________________________________________
tf_op_layer_Add_81 (TensorFlowO [(1, 24, 24, 1024)] 0 conv2d_80[0][0]
__________________________________________________________________________________________________
re_lu_77 (ReLU) (1, 24, 24, 1024) 0 tf_op_layer_Add_81[0][0]
__________________________________________________________________________________________________
conv2d_81 (Conv2D) (1, 24, 24, 1024) 1048576 re_lu_77[0][0]
__________________________________________________________________________________________________
tf_op_layer_Add_82 (TensorFlowO [(1, 24, 24, 1024)] 0 conv2d_81[0][0]
__________________________________________________________________________________________________
add_25 (Add) (1, 24, 24, 1024) 0 tf_op_layer_Add_82[0][0]
re_lu_75[0][0]
__________________________________________________________________________________________________
re_lu_78 (ReLU) (1, 24, 24, 1024) 0 add_25[0][0]
__________________________________________________________________________________________________
conv2d_82 (Conv2D) (1, 24, 24, 1024) 1048576 re_lu_78[0][0]
__________________________________________________________________________________________________
tf_op_layer_Add_83 (TensorFlowO [(1, 24, 24, 1024)] 0 conv2d_82[0][0]
__________________________________________________________________________________________________
re_lu_79 (ReLU) (1, 24, 24, 1024) 0 tf_op_layer_Add_83[0][0]
__________________________________________________________________________________________________
conv2d_83 (Conv2D) (1, 24, 24, 1024) 294912 re_lu_79[0][0]
__________________________________________________________________________________________________
tf_op_layer_Add_84 (TensorFlowO [(1, 24, 24, 1024)] 0 conv2d_83[0][0]
__________________________________________________________________________________________________
re_lu_80 (ReLU) (1, 24, 24, 1024) 0 tf_op_layer_Add_84[0][0]
__________________________________________________________________________________________________
conv2d_84 (Conv2D) (1, 24, 24, 1024) 1048576 re_lu_80[0][0]
__________________________________________________________________________________________________
tf_op_layer_Add_85 (TensorFlowO [(1, 24, 24, 1024)] 0 conv2d_84[0][0]
__________________________________________________________________________________________________
add_26 (Add) (1, 24, 24, 1024) 0 tf_op_layer_Add_85[0][0]
re_lu_78[0][0]
__________________________________________________________________________________________________
re_lu_81 (ReLU) (1, 24, 24, 1024) 0 add_26[0][0]
__________________________________________________________________________________________________
conv2d_85 (Conv2D) (1, 24, 24, 1024) 1048576 re_lu_81[0][0]
__________________________________________________________________________________________________
tf_op_layer_Add_86 (TensorFlowO [(1, 24, 24, 1024)] 0 conv2d_85[0][0]
__________________________________________________________________________________________________
re_lu_82 (ReLU) (1, 24, 24, 1024) 0 tf_op_layer_Add_86[0][0]
__________________________________________________________________________________________________
conv2d_86 (Conv2D) (1, 24, 24, 1024) 294912 re_lu_82[0][0]
__________________________________________________________________________________________________
tf_op_layer_Add_87 (TensorFlowO [(1, 24, 24, 1024)] 0 conv2d_86[0][0]
__________________________________________________________________________________________________
re_lu_83 (ReLU) (1, 24, 24, 1024) 0 tf_op_layer_Add_87[0][0]
__________________________________________________________________________________________________
conv2d_87 (Conv2D) (1, 24, 24, 1024) 1048576 re_lu_83[0][0]
__________________________________________________________________________________________________
tf_op_layer_Add_88 (TensorFlowO [(1, 24, 24, 1024)] 0 conv2d_87[0][0]
__________________________________________________________________________________________________
add_27 (Add) (1, 24, 24, 1024) 0 tf_op_layer_Add_88[0][0]
re_lu_81[0][0]
__________________________________________________________________________________________________
re_lu_84 (ReLU) (1, 24, 24, 1024) 0 add_27[0][0]
__________________________________________________________________________________________________
conv2d_88 (Conv2D) (1, 24, 24, 1024) 1048576 re_lu_84[0][0]
__________________________________________________________________________________________________
tf_op_layer_Add_89 (TensorFlowO [(1, 24, 24, 1024)] 0 conv2d_88[0][0]
__________________________________________________________________________________________________
re_lu_85 (ReLU) (1, 24, 24, 1024) 0 tf_op_layer_Add_89[0][0]
__________________________________________________________________________________________________
conv2d_89 (Conv2D) (1, 24, 24, 1024) 294912 re_lu_85[0][0]
__________________________________________________________________________________________________
tf_op_layer_Add_90 (TensorFlowO [(1, 24, 24, 1024)] 0 conv2d_89[0][0]
__________________________________________________________________________________________________
re_lu_86 (ReLU) (1, 24, 24, 1024) 0 tf_op_layer_Add_90[0][0]
__________________________________________________________________________________________________
conv2d_90 (Conv2D) (1, 24, 24, 1024) 1048576 re_lu_86[0][0]
__________________________________________________________________________________________________
tf_op_layer_Add_91 (TensorFlowO [(1, 24, 24, 1024)] 0 conv2d_90[0][0]
__________________________________________________________________________________________________
add_28 (Add) (1, 24, 24, 1024) 0 tf_op_layer_Add_91[0][0]
re_lu_84[0][0]
__________________________________________________________________________________________________
re_lu_87 (ReLU) (1, 24, 24, 1024) 0 add_28[0][0]
__________________________________________________________________________________________________
conv2d_91 (Conv2D) (1, 24, 24, 1024) 1048576 re_lu_87[0][0]
__________________________________________________________________________________________________
tf_op_layer_Add_92 (TensorFlowO [(1, 24, 24, 1024)] 0 conv2d_91[0][0]
__________________________________________________________________________________________________
re_lu_88 (ReLU) (1, 24, 24, 1024) 0 tf_op_layer_Add_92[0][0]
__________________________________________________________________________________________________
conv2d_92 (Conv2D) (1, 24, 24, 1024) 294912 re_lu_88[0][0]
__________________________________________________________________________________________________
tf_op_layer_Add_93 (TensorFlowO [(1, 24, 24, 1024)] 0 conv2d_92[0][0]
__________________________________________________________________________________________________
re_lu_89 (ReLU) (1, 24, 24, 1024) 0 tf_op_layer_Add_93[0][0]
__________________________________________________________________________________________________
conv2d_93 (Conv2D) (1, 24, 24, 1024) 1048576 re_lu_89[0][0]
__________________________________________________________________________________________________
tf_op_layer_Add_94 (TensorFlowO [(1, 24, 24, 1024)] 0 conv2d_93[0][0]
__________________________________________________________________________________________________
add_29 (Add) (1, 24, 24, 1024) 0 tf_op_layer_Add_94[0][0]
re_lu_87[0][0]
__________________________________________________________________________________________________
re_lu_90 (ReLU) (1, 24, 24, 1024) 0 add_29[0][0]
__________________________________________________________________________________________________
conv2d_94 (Conv2D) (1, 24, 24, 2048) 2097152 re_lu_90[0][0]
__________________________________________________________________________________________________
tf_op_layer_Add_95 (TensorFlowO [(1, 24, 24, 2048)] 0 conv2d_94[0][0]
__________________________________________________________________________________________________
re_lu_91 (ReLU) (1, 24, 24, 2048) 0 tf_op_layer_Add_95[0][0]
__________________________________________________________________________________________________
conv2d_95 (Conv2D) (1, 12, 12, 2048) 1179648 re_lu_91[0][0]
__________________________________________________________________________________________________
tf_op_layer_Add_96 (TensorFlowO [(1, 12, 12, 2048)] 0 conv2d_95[0][0]
__________________________________________________________________________________________________
re_lu_92 (ReLU) (1, 12, 12, 2048) 0 tf_op_layer_Add_96[0][0]
__________________________________________________________________________________________________
conv2d_96 (Conv2D) (1, 12, 12, 2048) 4194304 re_lu_92[0][0]
__________________________________________________________________________________________________
conv2d_97 (Conv2D) (1, 12, 12, 2048) 2097152 re_lu_90[0][0]
__________________________________________________________________________________________________
tf_op_layer_Add_97 (TensorFlowO [(1, 12, 12, 2048)] 0 conv2d_96[0][0]
__________________________________________________________________________________________________
tf_op_layer_Add_98 (TensorFlowO [(1, 12, 12, 2048)] 0 conv2d_97[0][0]
__________________________________________________________________________________________________
add_30 (Add) (1, 12, 12, 2048) 0 tf_op_layer_Add_97[0][0]
tf_op_layer_Add_98[0][0]
__________________________________________________________________________________________________
re_lu_93 (ReLU) (1, 12, 12, 2048) 0 add_30[0][0]
__________________________________________________________________________________________________
conv2d_98 (Conv2D) (1, 12, 12, 2048) 4194304 re_lu_93[0][0]
__________________________________________________________________________________________________
tf_op_layer_Add_99 (TensorFlowO [(1, 12, 12, 2048)] 0 conv2d_98[0][0]
__________________________________________________________________________________________________
re_lu_94 (ReLU) (1, 12, 12, 2048) 0 tf_op_layer_Add_99[0][0]
__________________________________________________________________________________________________
conv2d_99 (Conv2D) (1, 12, 12, 2048) 1179648 re_lu_94[0][0]
__________________________________________________________________________________________________
tf_op_layer_Add_100 (TensorFlow [(1, 12, 12, 2048)] 0 conv2d_99[0][0]
__________________________________________________________________________________________________
re_lu_95 (ReLU) (1, 12, 12, 2048) 0 tf_op_layer_Add_100[0][0]
__________________________________________________________________________________________________
conv2d_100 (Conv2D) (1, 12, 12, 2048) 4194304 re_lu_95[0][0]
__________________________________________________________________________________________________
tf_op_layer_Add_101 (TensorFlow [(1, 12, 12, 2048)] 0 conv2d_100[0][0]
__________________________________________________________________________________________________
add_31 (Add) (1, 12, 12, 2048) 0 tf_op_layer_Add_101[0][0]
re_lu_93[0][0]
__________________________________________________________________________________________________
re_lu_96 (ReLU) (1, 12, 12, 2048) 0 add_31[0][0]
__________________________________________________________________________________________________
conv2d_101 (Conv2D) (1, 12, 12, 2048) 4194304 re_lu_96[0][0]
__________________________________________________________________________________________________
tf_op_layer_Add_102 (TensorFlow [(1, 12, 12, 2048)] 0 conv2d_101[0][0]
__________________________________________________________________________________________________
re_lu_97 (ReLU) (1, 12, 12, 2048) 0 tf_op_layer_Add_102[0][0]
__________________________________________________________________________________________________
conv2d_102 (Conv2D) (1, 12, 12, 2048) 1179648 re_lu_97[0][0]
__________________________________________________________________________________________________
tf_op_layer_Add_103 (TensorFlow [(1, 12, 12, 2048)] 0 conv2d_102[0][0]
__________________________________________________________________________________________________
re_lu_98 (ReLU) (1, 12, 12, 2048) 0 tf_op_layer_Add_103[0][0]
__________________________________________________________________________________________________
conv2d_103 (Conv2D) (1, 12, 12, 2048) 4194304 re_lu_98[0][0]
__________________________________________________________________________________________________
tf_op_layer_Add_104 (TensorFlow [(1, 12, 12, 2048)] 0 conv2d_103[0][0]
__________________________________________________________________________________________________
add_32 (Add) (1, 12, 12, 2048) 0 tf_op_layer_Add_104[0][0]
re_lu_96[0][0]
__________________________________________________________________________________________________
re_lu_99 (ReLU) (1, 12, 12, 2048) 0 add_32[0][0]
__________________________________________________________________________________________________
conv2d_104 (Conv2D) (1, 12, 12, 256) 4718592 re_lu_99[0][0]
__________________________________________________________________________________________________
re_lu_100 (ReLU) (1, 12, 12, 256) 0 conv2d_104[0][0]
__________________________________________________________________________________________________
conv2d_107 (Conv2D) (1, 24, 24, 256) 2359296 re_lu_90[0][0]
__________________________________________________________________________________________________
conv2d_105 (Conv2D) (1, 12, 12, 256) 589824 re_lu_100[0][0]
__________________________________________________________________________________________________
re_lu_102 (ReLU) (1, 24, 24, 256) 0 conv2d_107[0][0]
__________________________________________________________________________________________________
tf_op_layer_Add_105 (TensorFlow [(1, 12, 12, 256)] 0 conv2d_105[0][0]
__________________________________________________________________________________________________
conv2d_108 (Conv2D) (1, 24, 24, 256) 589824 re_lu_102[0][0]
__________________________________________________________________________________________________
re_lu_101 (ReLU) (1, 12, 12, 256) 0 tf_op_layer_Add_105[0][0]
__________________________________________________________________________________________________
tf_op_layer_Add_107 (TensorFlow [(1, 24, 24, 256)] 0 conv2d_108[0][0]
__________________________________________________________________________________________________
conv2d_106 (Conv2D) (1, 12, 12, 256) 589824 re_lu_101[0][0]
__________________________________________________________________________________________________
re_lu_103 (ReLU) (1, 24, 24, 256) 0 tf_op_layer_Add_107[0][0]
__________________________________________________________________________________________________
tf_op_layer_Add_106 (TensorFlow [(1, 12, 12, 256)] 0 conv2d_106[0][0]
__________________________________________________________________________________________________
conv2d_109 (Conv2D) (1, 24, 24, 256) 589824 re_lu_103[0][0]
__________________________________________________________________________________________________
add_33 (Add) (1, 12, 12, 256) 0 tf_op_layer_Add_106[0][0]
re_lu_100[0][0]
__________________________________________________________________________________________________
tf_op_layer_Add_108 (TensorFlow [(1, 24, 24, 256)] 0 conv2d_109[0][0]
__________________________________________________________________________________________________
tf_op_layer_ResizeBilinear (Ten [(1, 24, 24, 256)] 0 add_33[0][0]
__________________________________________________________________________________________________
add_34 (Add) (1, 24, 24, 256) 0 tf_op_layer_Add_108[0][0]
re_lu_102[0][0]
__________________________________________________________________________________________________
add_35 (Add) (1, 24, 24, 256) 0 tf_op_layer_ResizeBilinear[0][0]
add_34[0][0]
__________________________________________________________________________________________________
re_lu_104 (ReLU) (1, 24, 24, 256) 0 add_35[0][0]
__________________________________________________________________________________________________
conv2d_112 (Conv2D) (1, 48, 48, 256) 1179648 re_lu_21[0][0]
__________________________________________________________________________________________________
conv2d_110 (Conv2D) (1, 24, 24, 256) 589824 re_lu_104[0][0]
__________________________________________________________________________________________________
re_lu_106 (ReLU) (1, 48, 48, 256) 0 conv2d_112[0][0]
__________________________________________________________________________________________________
tf_op_layer_Add_109 (TensorFlow [(1, 24, 24, 256)] 0 conv2d_110[0][0]
__________________________________________________________________________________________________
conv2d_113 (Conv2D) (1, 48, 48, 256) 589824 re_lu_106[0][0]
__________________________________________________________________________________________________
re_lu_105 (ReLU) (1, 24, 24, 256) 0 tf_op_layer_Add_109[0][0]
__________________________________________________________________________________________________
tf_op_layer_Add_111 (TensorFlow [(1, 48, 48, 256)] 0 conv2d_113[0][0]
__________________________________________________________________________________________________
conv2d_111 (Conv2D) (1, 24, 24, 256) 589824 re_lu_105[0][0]
__________________________________________________________________________________________________
re_lu_107 (ReLU) (1, 48, 48, 256) 0 tf_op_layer_Add_111[0][0]
__________________________________________________________________________________________________
tf_op_layer_Add_110 (TensorFlow [(1, 24, 24, 256)] 0 conv2d_111[0][0]
__________________________________________________________________________________________________
conv2d_114 (Conv2D) (1, 48, 48, 256) 589824 re_lu_107[0][0]
__________________________________________________________________________________________________
add_36 (Add) (1, 24, 24, 256) 0 tf_op_layer_Add_110[0][0]
re_lu_104[0][0]
__________________________________________________________________________________________________
tf_op_layer_Add_112 (TensorFlow [(1, 48, 48, 256)] 0 conv2d_114[0][0]
__________________________________________________________________________________________________
tf_op_layer_ResizeBilinear_1 (T [(1, 48, 48, 256)] 0 add_36[0][0]
__________________________________________________________________________________________________
add_37 (Add) (1, 48, 48, 256) 0 tf_op_layer_Add_112[0][0]
re_lu_106[0][0]
__________________________________________________________________________________________________
add_38 (Add) (1, 48, 48, 256) 0 tf_op_layer_ResizeBilinear_1[0][0
add_37[0][0]
__________________________________________________________________________________________________
re_lu_108 (ReLU) (1, 48, 48, 256) 0 add_38[0][0]
__________________________________________________________________________________________________
conv2d_117 (Conv2D) (1, 96, 96, 256) 589824 re_lu_9[0][0]
__________________________________________________________________________________________________
conv2d_115 (Conv2D) (1, 48, 48, 256) 589824 re_lu_108[0][0]
__________________________________________________________________________________________________
re_lu_110 (ReLU) (1, 96, 96, 256) 0 conv2d_117[0][0]
__________________________________________________________________________________________________
tf_op_layer_Add_113 (TensorFlow [(1, 48, 48, 256)] 0 conv2d_115[0][0]
__________________________________________________________________________________________________
conv2d_118 (Conv2D) (1, 96, 96, 256) 589824 re_lu_110[0][0]
__________________________________________________________________________________________________
re_lu_109 (ReLU) (1, 48, 48, 256) 0 tf_op_layer_Add_113[0][0]
__________________________________________________________________________________________________
tf_op_layer_Add_115 (TensorFlow [(1, 96, 96, 256)] 0 conv2d_118[0][0]
__________________________________________________________________________________________________
conv2d_116 (Conv2D) (1, 48, 48, 256) 589824 re_lu_109[0][0]
__________________________________________________________________________________________________
re_lu_111 (ReLU) (1, 96, 96, 256) 0 tf_op_layer_Add_115[0][0]
__________________________________________________________________________________________________
tf_op_layer_Add_114 (TensorFlow [(1, 48, 48, 256)] 0 conv2d_116[0][0]
__________________________________________________________________________________________________
conv2d_119 (Conv2D) (1, 96, 96, 256) 589824 re_lu_111[0][0]
__________________________________________________________________________________________________
add_39 (Add) (1, 48, 48, 256) 0 tf_op_layer_Add_114[0][0]
re_lu_108[0][0]
__________________________________________________________________________________________________
tf_op_layer_Add_116 (TensorFlow [(1, 96, 96, 256)] 0 conv2d_119[0][0]
__________________________________________________________________________________________________
tf_op_layer_ResizeBilinear_2 (T [(1, 96, 96, 256)] 0 add_39[0][0]
__________________________________________________________________________________________________
add_40 (Add) (1, 96, 96, 256) 0 tf_op_layer_Add_116[0][0]
re_lu_110[0][0]
__________________________________________________________________________________________________
add_41 (Add) (1, 96, 96, 256) 0 tf_op_layer_ResizeBilinear_2[0][0
add_40[0][0]
__________________________________________________________________________________________________
re_lu_112 (ReLU) (1, 96, 96, 256) 0 add_41[0][0]
__________________________________________________________________________________________________
conv2d_120 (Conv2D) (1, 96, 96, 256) 589824 re_lu_112[0][0]
__________________________________________________________________________________________________
tf_op_layer_Add_117 (TensorFlow [(1, 96, 96, 256)] 0 conv2d_120[0][0]
__________________________________________________________________________________________________
re_lu_113 (ReLU) (1, 96, 96, 256) 0 tf_op_layer_Add_117[0][0]
__________________________________________________________________________________________________
conv2d_121 (Conv2D) (1, 96, 96, 256) 589824 re_lu_113[0][0]
__________________________________________________________________________________________________
tf_op_layer_Add_118 (TensorFlow [(1, 96, 96, 256)] 0 conv2d_121[0][0]
__________________________________________________________________________________________________
add_42 (Add) (1, 96, 96, 256) 0 tf_op_layer_Add_118[0][0]
re_lu_112[0][0]
__________________________________________________________________________________________________
tf_op_layer_ResizeBilinear_3 (T [(1, 192, 192, 256)] 0 add_42[0][0]
__________________________________________________________________________________________________
conv2d_122 (Conv2D) (1, 192, 192, 128) 294912 tf_op_layer_ResizeBilinear_3[0][0
__________________________________________________________________________________________________
tf_op_layer_Add_119 (TensorFlow [(1, 192, 192, 128)] 0 conv2d_122[0][0]
__________________________________________________________________________________________________
tf_op_layer_ResizeBilinear_4 (T [(1, 384, 384, 128)] 0 tf_op_layer_Add_119[0][0]
__________________________________________________________________________________________________
conv2d_123 (Conv2D) (1, 384, 384, 32) 36864 tf_op_layer_ResizeBilinear_4[0][0
__________________________________________________________________________________________________
tf_op_layer_Add_120 (TensorFlow [(1, 384, 384, 32)] 0 conv2d_123[0][0]
__________________________________________________________________________________________________
re_lu_114 (ReLU) (1, 384, 384, 32) 0 tf_op_layer_Add_120[0][0]
__________________________________________________________________________________________________
conv2d_124 (Conv2D) (1, 384, 384, 1) 32 re_lu_114[0][0]
__________________________________________________________________________________________________
tf_op_layer_Add_121 (TensorFlow [(1, 384, 384, 1)] 0 conv2d_124[0][0]
__________________________________________________________________________________________________
re_lu_115 (ReLU) (1, 384, 384, 1) 0 tf_op_layer_Add_121[0][0]
__________________________________________________________________________________________________
tf_op_layer_Squeeze (TensorFlow [(1, 384, 384)] 0 re_lu_115[0][0]
__________________________________________________________________________________________________
tf_op_layer_inverse_depth (Tens [(1, 384, 384)] 0 tf_op_layer_Squeeze[0][0]
==================================================================================================
Total params: 103,976,160
Trainable params: 103,976,160
Non-trainable params: 0
__________________________________________________________________________________________________
WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow/python/training/tracking/tracking.py:111: Model.state_updates (from tensorflow.python.keras.engine.training) is deprecated and will be removed in a future version.
Instructions for updating:
This property should not be used in TensorFlow 2.0, as updates are applied automatically.
2020-10-20 22:16:47.075953: W tensorflow/python/util/util.cc:348] Sets are not currently considered sequences, but this may change in the future, so consider avoiding using them.
WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow/python/training/tracking/tracking.py:111: Layer.updates (from tensorflow.python.keras.engine.base_layer) is deprecated and will be removed in a future version.
Instructions for updating:
This property should not be used in TensorFlow 2.0, as updates are applied automatically.
Traceback (most recent call last):
File "openvino2tensorflow.py", line 788, in <module>
File "openvino2tensorflow.py", line 785, in main
File "openvino2tensorflow.py", line 704, in convert
File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/saved_model/save.py", line 1006, in save
path, saved_model.SerializeToString(deterministic=True))
ValueError: Message tensorflow.SavedModel exceeds maximum protobuf size of 2GB: 6958463348
The conversion to TFLite was successful. This is Float32, so it is a huge size of 416MB. I haven't checked the operation, so I don't know if I can infer correctly.
- Midasnet - TFLite Float32 - no quantization - model_float32.tflite
https://drive.google.com/file/d/1bvNiaYTx7SI8sVjrLOLe_vwg7sIwY3h0/view?usp=sharing
from pinto_model_zoo.
But, unfortunately, the protocol buffer size limit was exceeded and the timing of saving the model resulted in an error.
The conversion to TFLite was successful.
Do you mean that Keras-h5 model can't be saved, but Tflite was saved successfully?
Seems to be there is something wrong: https://colab.research.google.com/gist/AlexeyAB/c72d1c1ccb85e59c580725ada26072eb/tflite_midas_1.ipynb
/usr/local/lib/python3.6/dist-packages/tensorflow/lite/python/interpreter.py in allocate_tensors(self)
241 self._delegates = []
242 if experimental_delegates:
--> 243 self._delegates = experimental_delegates
244 for delegate in self._delegates:
245 self._interpreter.ModifyGraphWithDelegate(RuntimeError: tensorflow/lite/kernels/conv.cc:316 input->dims->data[3] != filter->dims->data[3] (256 != 8)Node number 5 (CONV_2D) failed to prepare.
from pinto_model_zoo.
@AlexeyAB
Thank you.
Do you mean that Keras-h5 model can't be saved, but Tflite was saved successfully?
Yes. It fails to save saved_model and h5.
Seems to be there is something wrong:
Hmmm. It's not easy.
Btw, I also tried converting EfficientNet-lite3, but it seems that the process after the last ReLU6 is not compatible with TFLite. I have not yet confirmed the operation of this one, too.
- tf_efficientnet_lite3_256x256.pb
https://drive.google.com/file/d/177ignW-Vz7yYATGTFP7G7utETedpgY8f/view?usp=sharing
from pinto_model_zoo.
@AlexeyAB
Conv2D groups - TFLite
tensorflow/tensorflow#40044
Unfortunately, it seems that the current situation is not supported.
from pinto_model_zoo.
@AlexeyAB
I implemented GoupConvolusion with the standard Conv2D and Split, Concat, although I may have failed to transpose the weights. This model is chaotic.
Midasnet - Float32 - GroupConvolusion - TFLite(.tflite)
https://drive.google.com/file/d/1YYp8z3uATTH6kgOdcjq2jPWpVtmqmFDN/view?usp=sharing
from pinto_model_zoo.
@PINTO0309
Thanks! Yes, it works, but it seems there is something wrong with weights (result at the end): https://colab.research.google.com/gist/AlexeyAB/6ce80cc647df7eb143a3b6bb7d0a1861/tflite_midas_1.ipynb
from pinto_model_zoo.
@AlexeyAB
I don't know if the conversion was successful, but the result looks good. Is this the result you were hoping for?
Since I have directly replaced the Google Drive model, you can simply rerun the Notebook you provided and it should produce the same results.
Please correct just one line below.
img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)# / 255.0
from pinto_model_zoo.
@PINTO0309
Great!
Yes, it is very similar to baseline result: https://colab.research.google.com/gist/AlexeyAB/24c52723fca01d351e45c4a4ed14af9c/tflite_midas_1.ipynb#scrollTo=9Q8CUqPztgYS
There is a difference, possibly due to slightly different models.
Also did you try to convert default classifier EfficientNet-lite?
from pinto_model_zoo.
@AlexeyAB
I'm going to start trying to convert EfficientNet-lite today.
Currently there seems to be a problem with Gather or Reshape or ShapeOf conversions, so I need to debug it.
from pinto_model_zoo.
What are the benefits of successfully completing this conversion process?
-
EfficientNet-Lite and ResNext cover the most of widely used models and operations, and currently the EfficientNet-Lite one of the most efficient (speed/accuracy) model for Embedded/Mobile CPU/GPU/NPU
-
Pytorch is widely used for training by developers and researchers, while Tensorflow is widely used for deployment Smartphones-TFLite/TPU-Edge/TFJS/TF-TRT/TF-serving/..., but
onnx_tf
work poorly, so another Pytorch -> TensorFlow converter is required
So your converter can be very useful https://github.com/PINTO0309/openvino2tensorflow.git
from pinto_model_zoo.
@AlexeyAB
I spent an hour or so going over the errors in the model, but I couldn't find any mistakes. I converted it to saved_model and then back-converted it to OpenVINO IR again to check the weights and structure. There seems to be no errors in the weight and structure, only the structure of the final processing part of my model seems to have been simply reconfigured.
from pinto_model_zoo.
@PINTO0309 Hi,
https://twitter.com/PINTO03091/status/1322723345838731265
Splitを実装するアイデアがなかなか浮かばないです… これができたらYoloV4 PyTorch が自動コンバートできるようになるんですけどね〜
What is the problem with 'split'? As I understand you successfully used split for Grouped Convolution.
What YoloV4 PyTorch repository do you mean? https://github.com/WongKinYiu/PyTorch_YOLOv4 or https://github.com/Tianxiaomo/pytorch-YOLOv4 or https://github.com/maudzung/Complex-YOLOv4-Pytorch or https://github.com/maudzung/YOLO3D-YOLOv4-PyTorch ?
from pinto_model_zoo.
@AlexeyAB
Yes, it wasn't hard to break down GroupConvolusion into Split and Concat. However, when I try to convert the ONNX in the following repository, there is a standalone Split and the number of Outputs is indeterminate, which makes my implementation difficult. This is because the process after a single Split is not necessarily concatenated.
https://github.com/Tianxiaomo/pytorch-YOLOv4
from pinto_model_zoo.
@PINTO0309 Hi, Thanks, Great!
Unfortunately, there is still a bug in the Reshape operation of the 5D tensor that causes YoloV4 and ShuffuleNet conversions to fail.
Are you about YOLOv4 or CSP-P5-P7 models? https://github.com/WongKinYiu/PyTorch_YOLOv4#pretrained-models--comparison
Where is this bug, is it in Pytorch, TFlite, or your script?
Can you please give a link to a line of code, where is the problem?
from pinto_model_zoo.
Are you about YOLOv4 or CSP-P5-P7 models?
I am testing using the models in the following repositories
https://github.com/Tianxiaomo/pytorch-YOLOv4
Where is this bug, is it in Pytorch, TFlite, or your script?
This is a bug in my openvino2tensorflow.
Can you please give a link to a line of code, where is the problem?
Of course. I'm trying every night, but it's hard to solve the problem. If you combine Reshape and Transpose, and the tensor to be transformed is 5D or 6D, the transposition operation is difficult. So far, I haven't come up with any good ideas.
For example, I feel that converting [1,256,13,13] to [1,256,13,1,13,1] would be a very complex operation in TensorFlow, as shown below.
from pinto_model_zoo.
Converting ONNX generated by the old branch master to .pb is successful, but converting it to tflite seems to cause an error.
Hmmm... It's troubling.
https://github.com/WongKinYiu/PyTorch_YOLOv4/tree/master
-
ONNX - yolov4-pacsp-s.onnx
https://drive.google.com/file/d/15NooGj6Aa8lEv5Rs6TalngQDXK0uTZ8W/view?usp=sharing -
OpenVINO IR
https://drive.google.com/file/d/1wXmTvSPtxo22GGJCN1JT6T35dcpX7ME6/view?usp=sharing -
pb (Transpose OP's transposition has been disabled and the state is deliberately buggy)
https://drive.google.com/file/d/1Nmkgs7ApyYq-K2tNfhO5kbkm0Yg9FzIL/view?usp=sharing
from pinto_model_zoo.
For example, I feel that converting [1,256,13,13] to [1,256,13,1,13,1] would be a very complex operation in TensorFlow, as shown below.
Yes, there are quite complex transformations here when objects from different branches are merged.
Error dump when converting to TensorFlow Lite
It seems that there is also an issue - TFlite doesn't support all TF operations.
Do you get the same issue with u5
branch? https://github.com/WongKinYiu/PyTorch_YOLOv4/tree/u5
from pinto_model_zoo.
Do you get the same issue with u5 branch?
I first tried to generate onnx from the u5 branch, but couldn't export to onnx in the first place. I'll try a few more things with the u5 branch.
from pinto_model_zoo.
I fixed a bug in openvino2tensorflow and succeeded in converting YOLOv4 to tflite, although I have not checked the operation of the conversion to be correct.
I used the onnx YOLOv4 below.
In anticipation of the conversion to the EdgeTPU model, the PReLU is deliberately changed to a combination of Maximum and Minimum.
from pinto_model_zoo.
@PINTO0309
Great!
In anticipation of the conversion to the EdgeTPU model, the PReLU is deliberately changed to a combination of Maximum and Minimum.
Is it because EdgeTPU doesn't support PReLU?
Can you try to convert yolov4x-mish.onnx
to yolov4x-mish.tflite
?
-
There is converter
yolov4x-mish.weights
toyolov4x-mish.onnx
https://github.com/linghu8812/tensorrt_inference/tree/master/Yolov4#2-darknet-model-zoo
from pinto_model_zoo.
@PINTO0309 Great!
I found that there is a bug regarding the Resize OP conversion in either edgetpu_compiler or TFLiteConverter.
How did you solve or avoid it?
It was very hard work, but it looks like I was able to refurbish openvino2tensorflow to generate the EdgeTPU model of YOLOv4-tiny.
Did you check it, does it produce approximately the same result as source yolov4-tiny model?
What source model do you use, is it yolov4-tiny?
Is it Pytorch URL or TensorFlow URL or Darknet URL or OpenVINO URL or TensorRT/ONNX URL yolov4-tiny model?
from pinto_model_zoo.
Hi @PINTO0309
I tried converting the keras model from https://github.com/bubbliiiing/yolov4-tiny-keras.git but had no luck in the end as I received model not quantized when passing the model to the edgetpu_compiler.
Here is the process I followed.
1- convert the keras model to frozen graph (.pb)
2- convert the .pb model to openvino using python mo_tf.py --input_model {pb_file} --output_dir {output_dir} --input_shape {input_shape_str}
3- openvino2tensorflow --model_path={model_path} --output_weight_quant_tflite True
4- Run the edge_tpu compiler on the resulting file edgetpu_compiler model_weight_quant.tflite
Is there anything I'm missing here?
Here is a glance to the output model file.
from pinto_model_zoo.
Since output_weight_quant_tflite
only quantizes the weights, it cannot be converted to the edgetpu model. edgetpu needs to quantize all the OPs and perform Full Integer Quantization, so it must be converted according to the following procedure.
$ openvino2tensorflow \
--model_path={model_path} \
--output_saved_model True
- quantization.py
import tensorflow as tf
import tensorflow_datasets as tfds
import numpy as np
def representative_dataset_gen():
for data in raw_test_data.take(10):
image = data['image'].numpy()
image = tf.image.resize(image, (416, 416))
image = image[np.newaxis,:,:,:]
image = image / 255.
yield [image]
raw_test_data, info = tfds.load(name="voc/2007", with_info=True, split="validation", data_dir="~/TFDS", download=True)
# Full Integer Quantization - Input/Output=float32
converter = tf.lite.TFLiteConverter.from_saved_model('saved_model')
converter.optimizations = [tf.lite.Optimize.DEFAULT]
converter.target_spec.supported_ops = [tf.lite.OpsSet.TFLITE_BUILTINS_INT8,tf.lite.OpsSet.SELECT_TF_OPS]
converter.representative_dataset = representative_dataset_gen
converter.inference_input_type = tf.uint8
converter.inference_output_type = tf.uint8
tflite_quant_model = converter.convert()
with open('yolov4_416_full_integer_quant.tflite', 'wb') as w:
w.write(tflite_quant_model)
print("Full Integer Quantization complete! - yolov4_416_full_integer_quant.tflite")
$ python3 quantization.py
$ edgetpu_compiler -s yolov4_416_full_integer_quant.tflite
The normalization process of representative_dataset_gen
should be adjusted by you.
from pinto_model_zoo.
Hi @PINTO0309
Thanks for your response. I am having issues converting the model, wondering if you can help?
This time I:
- Downloaded
yolov4_tiny_voc_416x416_FP32
from your repo - Used
openvino/ubuntu18_dev
docker image and installedpip3 install openvino2tensorflow --upgrade
openvino2tensorflow --model_path=yolov4_tiny_voc_416x416_FP32.xml --output_saved_model True
- Convert the resulting
saved_model
usingquantization.py
The error I get is:
Traceback (most recent call last):
File "quant.py", line 21, in <module>
converter = tf.lite.TFLiteConverter.from_saved_model('saved_model')
File "/usr/local/lib/python3.6/dist-packages/tensorflow/lite/python/lite.py", line 1069, in from_saved_model
saved_model = _load(saved_model_dir, tags)
File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/saved_model/load.py", line 859, in load
return load_internal(export_dir, tags, options)["root"]
File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/saved_model/load.py", line 909, in load_internal
root = load_v1_in_v2.load(export_dir, tags)
File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/saved_model/load_v1_in_v2.py", line 279, in load
return loader.load(tags=tags)
File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/saved_model/load_v1_in_v2.py", line 204, in load
meta_graph_def = self.get_meta_graph_def_from_tags(tags)
File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/saved_model/load_v1_in_v2.py", line 87, in get_meta_graph_def_from_tags
tags)
File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/saved_model/loader_impl.py", line 358, in get_meta_graph_def_from_tags
"\navailable_tags: " + str(available_tags))
RuntimeError: MetaGraphDef associated with tags {'serve'} could not be found in SavedModel. To inspect available tag-sets in the SavedModel, please use the SavedModel CLI: `saved_model_cli`
available_tags: []
I tried different tensorflow version but can't get passed this error
I would appreciate your help
from pinto_model_zoo.
@itsmasabdi
You should use the Docker environment.
https://github.com/PINTO0309/openvino2tensorflow#4-setup
from pinto_model_zoo.
Related Issues (20)
- Link is not working for Hitnet HOT 1
- Rapid wrong initial onxx model HOT 1
- Add MiVOLO
- Explaination on The YoloX Body-Head-Hand output dimension? HOT 2
- Add HFGI
- ONNX version for person-reidentification-retail-0277 HOT 5
- Zoedepth ONNX conversion Script HOT 3
- gfpgan coreml model HOT 1
- License of RAFT models HOT 3
- Script to convert RAFT models HOT 3
- Blazeface onnx model HOT 1
- BodyPix on MacOS - Dilation not supported for AutoPadType::SAME_UPPER or AutoPadType::SAME_LOWER HOT 8
- TOPK operator for RKNN export HOT 1
- InstructIR
- 064_Dense_Depth seems to have wrong dimensions HOT 1
- Midas2 model on coral edge TPU HOT 1
- Difference on model outputs (tflite, openvino IR, and Onnx) in model 227_face-detection-adas-0001 HOT 1
- bad results for 342_ALIKE HOT 1
- Aborted (core dumped) for full quantized tinyhitnet model
- 091_gaze-estimation-adas-0002 network HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from pinto_model_zoo.