ethereon / caffe-tensorflow Goto Github PK

During graph construction, network should not be able to know the batch size (the first dimension of tensor). I have made some modifications to deal with this problem, not sure if you need to support this feature.

I'm trying to convert a NIN model and post conversion while saving source i get this error:
"Error encountered: No handler found for node kind: SoftmaxWithLoss (expected: map_softmax_with_loss)"
Is that a problem with the SoftmaxWithLoss support or is this a problem with my specific model?
thanks
S.R.H

The inception net in the Android Tensorflow Demo stores the model as a protobuf file (tensorflow_inception_graph.pb). Is it possible to convert the python scripts outputted to protobuf format?

I have what I imagine is a rather simple problem. I'm trying to import the standard bvlc_alexnet model from the Caffe github (caffe/models/bvlc_alexnet). I've tried both the train_val and the solver prototext file, and get different errors for each - both in the first layer.

In train_val.prototext, the first layer is:

layer {
  name: "data"
  type: "Data"
  top: "data"
  top: "label"
  include {
    phase: TRAIN
  }
  transform_param {
    mirror: true
    crop_size: 227
    mean_file: "data/ilsvrc12/imagenet_mean.binaryproto"
  }
  data_param {
    source: "examples/imagenet/ilsvrc12_train_lmdb"
    batch_size: 256
    backend: LMDB
  }
}

and the error is "Cannot determine the dimensions of data layer".
If I use deploy.prototext:

layer {
  name: "data"
  type: "Input"
  top: "data"
  input_param { shape: { dim: 10 dim: 3 dim: 227 dim: 227 } }
}

I get an error because the "Input" layer is not recognized, and also because the "input_param" property is not recognized.

I'm sure I'm doing something foolish, but can't seem to figure it out! Perhaps including a sample prototext file that converts successfully would help people like me massage our files into the correct format?

Hi,
I tried to convert the VGG16 weights trained from Places365 dataset to tensorflow. But I get the following error:

kang@herakles:~/Documents/work_code_PC1/caffe-tensorflow$ python convert.py deploy_vgg16_places365.prototxt --caffemodel vgg16_places365.caffemodel --data-output-path placesnet.npy --code-output-path placesnet.py
I tensorflow/stream_executor/dso_loader.cc:105] successfully opened CUDA library libcublas.so locally
I tensorflow/stream_executor/dso_loader.cc:105] successfully opened CUDA library libcudnn.so locally
I tensorflow/stream_executor/dso_loader.cc:105] successfully opened CUDA library libcufft.so locally
I tensorflow/stream_executor/dso_loader.cc:105] successfully opened CUDA library libcuda.so.1 locally
I tensorflow/stream_executor/dso_loader.cc:105] successfully opened CUDA library libcurand.so locally
python: ./google/protobuf/generated_message_util.h:80: const string& google::protobuf::internal::GetEmptyStringAlreadyInited(): Assertion `empty_string_ != __null' failed.
Aborted (core dumped)

How to fix that?
Thank you very much.

Hi,
I would like to know If it is possible to use the output of a certain stage of converted network as an input to another network ?
Which can then be trained and saved.
For Eg. I would like to use reproduce paper http://arxiv.org/abs/1411.4389 .
As the input I use their given singleframe_RGB model given in http://www.eecs.berkeley.edu/~lisa_anne/LRCN_video_weights.html .

when i use kaffe-tensorflow to convert (Network in Network ) to tensorflow,but when i load the converted net ,i get error,because avg_pool.shape (batch ,10,1,1) have not been fatten，and the next layer is softmax.(batch,10) so i add a flatten layer in network.py

@layer
    def flatten(self,input,name):
        dim = 1
        for d in input.get_shape()[1:].as_list():
                dim *= d
        return tf.reshape(input,[-1,dim])

I've been trying to load the standard VGG16 models and prototxt, but am getting the following error when trying to load the very first layer:

Unknown layer type encountered: None

The error is thrown in core.py:GraphBuilder:make_node, which has the following lines:

   kind = NodeKind.map_raw_kind(layer.type)
   if kind is None:
      raise KaffeError('Unknown layer type encountered: %s'%kind)

layer.type here is an integer (4 – no idea where that's coming from), but NodeKind.map_raw_kind is trying to match the layer name (e.g. "Convolution" to LAYER_TYPES, which contains a list of Caffe layer types). Naturally, 4 isn't in LAYER_TYPES, so it throws the error.

I can't figure out where this 4, instead of "Convolution", is coming from – tracing that back led me to kaffe/caffepb, which I don't understand. The layer object looks like this: <kaffe.caffepb.V1LayerParameter object at 0x1064c2a28>, and this is what print(layer) outputs right before the call to layer.type returns 4:

bottom: "data"
top: "conv1_1"
name: "conv1_1"
type: CONVOLUTION
convolution_param {
  num_output: 64
  pad: 1
  kernel_size: 3
}

Any thoughts as to what's going on? I imagine the wrong thing is being assigned to type, but I can't figure out where.

As an aside, the names in LAYER_DESCRIPTORS are all in CamelCase, while the ones in the official prototxts are in all-caps – I've modified manually as a quick fix, but might submit a .lower() fix if I have time today.

Hi, is there a plan to convert tensorflow models to caffe?

Hi,

thanks for the library. Is there a way to get the value of a specific layer? for example if I want to get feature vectors like fc7 or fc8, how should I get the values of these variables?
Or is it possible to run the model not to the end? for example have forward path just for first 4 layers?

Thanks

Hi All :
When I run
./convert.py alexnet.prototxt --caffemodel alexnet.caffemodel --data-output-path=alexTensorflow.py
after download alexnet model from Caffe, it appears error as following:
How to modify the code ?
Thanks

Traceback (most recent call last):
  File "./convert.py", line 60, in <module>
    main()
  File "./convert.py", line 56, in main
    args.phase)
  File "./convert.py", line 27, in convert
    transformer = TensorFlowTransformer(def_path, caffemodel_path, phase=phase)
  File "/home/larson/atensorflow_code/Caffe2Tensorflow/kaffe/tensorflow/transformer.py", line 221, in __init__
    self.load(def_path, data_path, phase)
  File "/home/larson/atensorflow_code/Caffe2Tensorflow/kaffe/tensorflow/transformer.py", line 227, in load
    graph = GraphBuilder(def_path, phase).build()
  File "/home/larson/atensorflow_code/Caffe2Tensorflow/kaffe/graph.py", line 140, in __init__
    self.load()
  File "/home/larson/atensorflow_code/Caffe2Tensorflow/kaffe/graph.py", line 146, in load
    text_format.Merge(def_file.read(), self.params)
  File "/usr/local/lib/python2.7/dist-packages/google/protobuf/text_format.py", line 309, in Merge
    return MergeLines(text.split('\n'), message, allow_unknown_extension)
  File "/usr/local/lib/python2.7/dist-packages/google/protobuf/text_format.py", line 346, in MergeLines
    _ParseOrMerge(lines, message, True, allow_unknown_extension)
  File "/usr/local/lib/python2.7/dist-packages/google/protobuf/text_format.py", line 371, in _ParseOrMerge
    allow_unknown_extension)
  File "/usr/local/lib/python2.7/dist-packages/google/protobuf/text_format.py", line 473, in _MergeField
    allow_unknown_extension)
  File "/usr/local/lib/python2.7/dist-packages/google/protobuf/text_format.py", line 493, in _MergeField
    _MergeScalarField(tokenizer, message, field, allow_multiple_scalars)
  File "/usr/local/lib/python2.7/dist-packages/google/protobuf/text_format.py", line 625, in _MergeScalarField
    value = tokenizer.ConsumeString()
  File "/usr/local/lib/python2.7/dist-packages/google/protobuf/text_format.py", line 901, in ConsumeString
    the_bytes = self.ConsumeByteString()
  File "/usr/local/lib/python2.7/dist-packages/google/protobuf/text_format.py", line 916, in ConsumeByteString
    the_list = [self._ConsumeSingleByteString()]
  File "/usr/local/lib/python2.7/dist-packages/google/protobuf/text_format.py", line 935, in _ConsumeSingleByteString
    raise self._ParseError('Expected string but found: %r' % (text,))
google.protobuf.text_format.ParseError: 9:9 : Expected string but found: 'Convolution'

The following is alexnet.prototxt

name: "CaffeNet"
input: "data"
input_dim: 1
input_dim: 3
input_dim: 227
input_dim: 227
layer {
  name: "conv1"
  type: Convolution
  bottom: "data"
  top: "conv1"
  blobs_lr: 1
  blobs_lr: 2
  weight_decay: 1
  weight_decay: 0
  convolution_param {
    num_output: 96
    kernel_size: 11
    stride: 4
  }
}
layer {
  name: "relu1"
  type: ReLU
  bottom: "conv1"
  top: "conv1"
}
layer {
  name: "pool1"
  type: Pooling
  bottom: "conv1"
  top: "pool1"
  pooling_param {
    pool: MAX
    kernel_size: 3
    stride: 2
  }
}
layer {
  name: "norm1"
  type: LRN
  bottom: "pool1"
  top: "norm1"
  lrn_param {
    local_size: 5
    alpha: 0.0001
    beta: 0.75
  }
}
layer {
  name: "conv2"
  type: Convolution
  bottom: "norm1"
  top: "conv2"
  blobs_lr: 1
  blobs_lr: 2
  weight_decay: 1
  weight_decay: 0
  convolution_param {
    num_output: 256
    pad: 2
    kernel_size: 5
    group: 2
  }
}
layer {
  name: "relu2"
  type: ReLU
  bottom: "conv2"
  top: "conv2"
}
layer {
  name: "pool2"
  type: Pooling
  bottom: "conv2"
  top: "pool2"
  pooling_param {
    pool: MAX
    kernel_size: 3
    stride: 2
  }
}
layer {
  name: "norm2"
  type: LRN
  bottom: "pool2"
  top: "norm2"
  lrn_param {
    local_size: 5
    alpha: 0.0001
    beta: 0.75
  }
}
layer {
  name: "conv3"
  type: Convolution
  bottom: "norm2"
  top: "conv3"
  blobs_lr: 1
  blobs_lr: 2
  weight_decay: 1
  weight_decay: 0
  convolution_param {
    num_output: 384
    pad: 1
    kernel_size: 3
  }
}
layer {
  name: "relu3"
  type: ReLU
  bottom: "conv3"
  top: "conv3"
}
layer {
  name: "conv4"
  type: Convolution
  bottom: "conv3"
  top: "conv4"
  blobs_lr: 1
  blobs_lr: 2
  weight_decay: 1
  weight_decay: 0
  convolution_param {
    num_output: 384
    pad: 1
    kernel_size: 3
    group: 2
  }
}
layer {
  name: "relu4"
  type: ReLU
  bottom: "conv4"
  top: "conv4"
}
layer {
  name: "conv5"
  type: Convolution
  bottom: "conv4"
  top: "conv5"
  blobs_lr: 1
  blobs_lr: 2
  weight_decay: 1
  weight_decay: 0
  convolution_param {
    num_output: 256
    pad: 1
    kernel_size: 3
    group: 2
  }
}
layer {
  name: "relu5"
  type: ReLU
  bottom: "conv5"
  top: "conv5"
}
layer {
  name: "pool5"
  type: Pooling
  bottom: "conv5"
  top: "pool5"
  pooling_param {
    pool: MAX
    kernel_size: 3
    stride: 2
  }
}
layer {
  name: "fc6"
  type: InnerProduct
  bottom: "pool5"
  top: "fc6"
  blobs_lr: 1
  blobs_lr: 2
  weight_decay: 1
  weight_decay: 0
  inner_product_param {
    num_output: 4096
  }
}
layer {
  name: "relu6"
  type: ReLU
  bottom: "fc6"
  top: "fc6"
}
layer {
  name: "fc7"
  type: InnerProduct
  bottom: "fc6"
  top: "fc7"
  blobs_lr: 1
  blobs_lr: 2
  weight_decay: 1
  weight_decay: 0
  inner_product_param {
    num_output: 4096
  }
}
layer {
  name: "relu7"
  type: ReLU
  bottom: "fc7"
  top: "fc7"
}
layer {
  name: "fc8_sbt"
  type: InnerProduct
  bottom: "fc7"
  top: "fc8_sbt"
  blobs_lr: 1
  blobs_lr: 2
  weight_decay: 1
  weight_decay: 0
  inner_product_param {
    num_output: 5
  }
}
layer {
  name: "prob"
  type: Softmax
  bottom: "fc8_sbt"
  top: "prob"
}

I used caffe-tensorflow to convert Caffe Alex network.
The convolution weights data format isn't as expected.

Convolution Node data_format value is 'NHWC' (default)
But the weights node data format was HWCN,
Looking at the "Shape" attribute values I found that it also formatted as HWCN .
For example, for conv1 layer

• expected W=11,H=11, C=3, N=96
• shape dim is {11,11,3,96}

I expected it to be {96,11,11,3} - NHWC

On the other hand, the above format matches the python convert code (network.py - line 123)

kernel = self.make_var('weights', shape=[k_h, k_w, c_i / group, c_o])

why does the converter script use this data format (HWCN) ?

This project doesn't have license file. Is it public domain licensed?

I wonder what is the best way to use the dropout layer in the Network class if I want to load a pretrained Caffe model and fine tune on another task with dropout. I found that the converted network does not contains dropout layers.

Currently I just define a keep_prob variable and manually insert dropout layers into the converted network, which is kind of messy. What is the expected way of using dropout layer in your original design?

Usually during training add a normalization to the weights used in CNN is a good way to improve the generalization performance.
If I did not misread the code, at the moment it seems that it is not trivial to get the weights during learning.
It seems to me that it might be a good idea to store also the weights in the Network class for easy retrieval during learning.

Hi, thank you for making public this project.
I was trying to run convert.py to get TensorFlow version of VGG16 model, but it raises a weird error.

The problem seems like, the value of layer.type is always in number (4: Convolution, 18: something else...), rather than the string like "Convolution" or "Pooling" (line #200 in kaffe/core.py).

I am not sure if I am doing something wrong...
Could you please help me get over this?

Thank you
-Taeksoo

Tensorflow has a new higher level abstraction similar to Lasagne (for Theano) now. It would be great if the conversion to tensorflow can use this new format to make the models much more neat/slim.
https://github.com/tensorflow/models/tree/master/inception/inception/slim

There seems to be an incoherence which prevents me from loading the alexnet weights. I get the following output. As you can see, the output of conv1 is 96 channels, but the input of conv 2 is 48 channels. It does the same with the reference or the alexnet network.

WARNING: PyCaffe not found! 
Falling back to protocol buffer implementation. 
This may take a couple of minutes.
Type                 Name                                          Param               Output
----------------------------------------------------------------------------------------------
Data                 data                                             --    (10, 3, 227, 227)
Convolution          conv1                               (96, 3, 11, 11)     (10, 96, 55, 55)
Pooling              pool1                                            --     (10, 96, 27, 27)
LRN                  norm1                                            --     (10, 96, 27, 27)
Convolution          conv2                               (256, 48, 5, 5)    (10, 256, 27, 27)
Pooling              pool2                                            --    (10, 256, 13, 13)
LRN                  norm2                                            --    (10, 256, 13, 13)
Convolution          conv3                              (384, 256, 3, 3)    (10, 384, 13, 13)
Convolution          conv4                              (384, 192, 3, 3)    (10, 384, 13, 13)
Convolution          conv5                              (256, 192, 3, 3)    (10, 256, 13, 13)
Pooling              pool5                                            --      (10, 256, 6, 6)
InnerProduct         fc6                                    (4096, 9216)     (10, 4096, 1, 1)
InnerProduct         fc7                                    (4096, 4096)     (10, 4096, 1, 1)
InnerProduct         fc8                                    (1000, 4096)     (10, 1000, 1, 1)
Softmax              prob                                             --     (10, 1000, 1, 1)
Converting data...
Saving data...
Saving source...
Done.

With the new updates of this repo, I got the Segmentation fault for a very simple command

./convert.py examples/mnist/lenet.prototxt --code-output-path=mynet.py

This does not happen when I use your repo cloned before May 2.

Hi,

I modified the finetune_mnist.py by importing the .npy model as well as the corresponding weight converted from several caffemodels(including VGG16, AlexNet, GoogLenet, etc.) using convert.py. However, all of the coverted models except the original LeNet generate a traceback error like follow(The following used AlexNet model):

Traceback (most recent call last):
File "finetune_mnist.py", line 34, in
net = MyNet({'data': images})
File "/home/username/Documents/caffe-tensorflow/examples/mnist/kaffe/tensorflow/network.py", line 43, in init
self.setup()
File "/home/username/Documents/caffe-tensorflow/examples/mnist/mynet.py", line 10, in setup
.conv(5, 5, 256, 1, 1, group=2, name='conv2')
File "/home/username/Documents/caffe-tensorflow/examples/mnist/kaffe/tensorflow/network.py", line 21, in layer_decorated
layer_output = op(self, layer_input, _args, *_kwargs)
File "/home/username/Documents/caffe-tensorflow/examples/mnist/kaffe/tensorflow/network.py", line 131, in conv
output_groups = [convolve(i, k) for i, k in zip(input_groups, kernel_groups)]
File "/home/username/Documents/caffe-tensorflow/examples/mnist/kaffe/tensorflow/network.py", line 121, in
convolve = lambda i, k: tf.nn.conv2d(i, k, [1, s_h, s_w, 1], padding=padding)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/gen_nn_ops.py", line 394, in conv2d
data_format=data_format, name=name)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/op_def_library.py", line 704, in apply_op
op_def=op_def)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/ops.py", line 2262, in create_op
set_shapes_for_outputs(ret)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/ops.py", line 1702, in set_shapes_for_outputs
shapes = shape_func(op)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/common_shapes.py", line 246, in conv2d_shape
padding)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/common_shapes.py", line 184, in get2d_conv_output_size
(row_stride, col_stride), padding_type)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/common_shapes.py", line 149, in get_conv_output_size
"Filter: %r Input: %r" % (filter_size, input_size))
ValueError: Filter must not be larger than the input: Filter: (5, 5) Input: (2, 2)

where the coverted AlexNet.py tensorflow code is as follow:

class AlexNet(Network):
def setup(self):
(self.feed('data')
.conv(11, 11, 96, 4, 4, padding='VALID', name='conv1')
.lrn(2, 2e-05, 0.75, name='norm1')
.max_pool(3, 3, 2, 2, padding='VALID', name='pool1')
.conv(5, 5, 256, 1, 1, group=2, name='conv2')
.lrn(2, 2e-05, 0.75, name='norm2')
.max_pool(3, 3, 2, 2, padding='VALID', name='pool2')
.conv(3, 3, 384, 1, 1, name='conv3')
.conv(3, 3, 384, 1, 1, group=2, name='conv4')
.conv(3, 3, 256, 1, 1, group=2, name='conv5')
.max_pool(3, 3, 2, 2, padding='VALID', name='pool5')
.fc(4096, name='fc6')
.fc(4096, name='fc7')
.fc(1000, relu=False, name='fc8')
.softmax(name='prob'))

I would like to know what might be the possible reason causing this common 'filter larger than input size' error and how could I fix it?

Thanks!

I tried to convert caffe's bvlc_googlenet, using the following script:
./convert.py CAFFE_ROOT_PATH/models/bvlc_googlenet/train_val.prototxt --code-output-path ./bvlc_googlenet.py
Then I got:
Error encountered: Multiple top nodes are not supported.
I am using the latest caffe version.
Where was i doing wrong? Thanks in advance.

Using the command: ./convert.py VGG_ILSVRC_16_layers_deploy.prototxt --code-output-path=mynet.py

.prototxt obtained from: https://gist.github.com/ksimonyan/211839e770f7b538e2d8

Hi,
I tried to convert the SqueezeNet weights trained to tensorflow.
But i get some errors:

ValueError: Unable to determine kernel parameter!

How to fix that?
Thank you very much!

Hi,

When I run the examples/imagenet/validate.py, I keep getting the following error at the final iteration:
E tensorflow/core/client/tensor_c_api.cc:485] FIFOQueue '_0_path_queue' is closed and has insufficient elements (requested 1, current size 0)
[[Node: path_queue_Dequeue = QueueDequeue_class=["loc:@path_queue"], component_types=[DT_INT32, DT_BOOL, DT_STRING], timeout_ms=-1, _device="/job:localhost/replica:0/task:0/cpu:0"]]

Does anyone know how to solve it?

Thanks!

I have convert the vgg-16 and vgg-19 caffemodel to tensorflow model, but i get one error about using the tensorflow model classify the image. vgg-16 get the conv5_1/weights does not exist, and vgg-19 get the conv5_4/weights does not exist. can you give me some advice?

Awesome work, thanks for putting it together! With that said, are there any plans for supporting batch normalization?

Hi,
I'm using this on AWS to convert a handtuned caffe model to tensorflow. CUDA is installed, Caffe also with Python support. Tensorflow also.

The problem is very strange : at the first time I launch python, I can import caffe succesfully. But after, I try to run convert.py, it fails and I'm not able to import caffe anymore in any python command line.

Logs :

ubuntu@ip:~$ python
Python 2.7.6 (default, Jun 22 2015, 17:58:13)
[GCC 4.8.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.

import caffe
import tensorflow
I tensorflow/stream_executor/dso_loader.cc:108] successfully opened CUDA library libcublas.so locally
I tensorflow/stream_executor/dso_loader.cc:108] successfully opened CUDA library libcudnn.so locally
I tensorflow/stream_executor/dso_loader.cc:108] successfully opened CUDA library libcufft.so locally
I tensorflow/stream_executor/dso_loader.cc:108] successfully opened CUDA library libcuda.so.1 locally
I tensorflow/stream_executor/dso_loader.cc:108] successfully opened CUDA library libcurand.so locally
print tensorflow.__version__
0.10.0rc0
ubuntu@ip:~/tensorflow/caffe-tensorflow$ python convert.py ../../caffe-master/models/deploy.prototxt --caffemodel ../../caffe-master/models/imagenet_train_iter_5000.caffemodel --data-output-path output/
I tensorflow/stream_executor/dso_loader.cc:108] successfully opened CUDA library libcublas.so locally
I tensorflow/stream_executor/dso_loader.cc:108] successfully opened CUDA library libcudnn.so locally
I tensorflow/stream_executor/dso_loader.cc:108] successfully opened CUDA library libcufft.so locally
I tensorflow/stream_executor/dso_loader.cc:108] successfully opened CUDA library libcuda.so.1 locally
I tensorflow/stream_executor/dso_loader.cc:108] successfully opened CUDA library libcurand.so locally

---

WARNING: PyCaffe not found!
Falling back to a pure protocol buffer implementation.
* Conversions will be drastically slower.
* This backend is UNTESTED!

---

Error encountered: Unknown layer type encountered: 4
ubuntu@ip:~/tensorflow/caffe-tensorflow$ python
Python 2.7.6 (default, Jun 22 2015, 17:58:13)
[GCC 4.8.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.

import caffe
Traceback (most recent call last):
File "", line 1, in
ImportError: No module named caffe

Hi.

I am trying to convert this model to tensorflow . Unfortunately I am getting the following error:

Error encountered: Unknown layer type encountered: Reduction

That layer is not supported?.

BTW, great project.

I want to convert googlenet v2 to tensorflow code. The traceback is shown as follows:

  $ ./convert.py train_val.prototxt googlenet_bn.caffemodel data.out source.out

Traceback (most recent call last):
  File "./convert.py", line 35, in <module>
    main()
  File "./convert.py", line 19, in main
    transformer = TensorFlowTransformer(args.def_path, args.data_path, phase=args.phase)
  File "/home/zhufengda/caffe-tensorflow/kaffe/tensorflow/transformer.py", line 188, in __init__
    self.load(def_path, data_path, phase)
  File "/home/zhufengda/caffe-tensorflow/kaffe/tensorflow/transformer.py", line 192, in load
    self.graph = GraphBuilder(def_path, data_path, phase).build()
  File "/home/zhufengda/caffe-tensorflow/kaffe/core.py", line 250, in __init__
    self.load()
  File "/home/zhufengda/caffe-tensorflow/kaffe/core.py", line 255, in load
    text_format.Merge(def_file.read(), self.params)
  File "/usr/lib/python2.7/site-packages/google/protobuf/text_format.py", line 309, in Merge
    return MergeLines(text.split('\n'), message, allow_unknown_extension)
  File "/usr/lib/python2.7/site-packages/google/protobuf/text_format.py", line 346, in MergeLines
    _ParseOrMerge(lines, message, True, allow_unknown_extension)
  File "/usr/lib/python2.7/site-packages/google/protobuf/text_format.py", line 371, in _ParseOrMerge
    allow_unknown_extension)
  File "/usr/lib/python2.7/site-packages/google/protobuf/text_format.py", line 443, in _MergeField
    message_descriptor.full_name, name))
google.protobuf.text_format.ParseError: 1:1 : Message type "caffe.NetParameter" has no field named "mean_value".

The only key that named mean_value is the first three lines of train_val.prototxt

I'm trying to convert a GoogLeNet with a MemoryData layer as the first layer:

layer {
  name: "googlenet"
  type: "MemoryData"
  top: "data"
  top: "label"
  memory_data_param {
    batch_size: 32
    channels: 3
    height: 224
    width: 224
  }
}

When I run convert.py, I run into the shape_identity issue:

# We most likely have a data layer on our hands. The problem is,
# Caffe infers the dimensions of the data from the source (eg: LMDB).
# We want to avoid reading datasets here. Fail for now.
# This can be temporarily fixed by transforming the data layer to
# Caffe's "input" layer (as is usually used in the "deploy" version).
# TODO: Find a better solution for this.

What does it mean to transform the data layer to an input layer? Can you provide more information on how to do this? Thanks.

I trained GoogleNet (v1) with BatchNorm on Caffe. Prediction is OK on Caffe. When converting to TF model, prediction results are quite random and unpredictable! Then I trained another GoogleNet (v1) without BatchNorm layers on Caffe. Now predictions are OK on both Caffe and the converted TF model. So I think there may be some bugs relevant to BatchNorm part. Have you tested it?

Best,

I tried to convert caffe model 'VGG_CNN_M' to fine-tune in Tensorflow. However, an error occurred:
'Unknown layer type encountered:4'
Do I need to install Caffe first, or how come is the error.
Thanks!

Can you provide numpy blobs, so that it can be used without Caffe?
Also I am trying to convert VGG-Face net from lua code, it should have similar structure like this?

Maybe we should merge our ideas:
https://github.com/pannous/tensor-caffe

When I run classify.py and test on GoogleNet, the result is not as expected.
When I tested on a dataset with more than 2000 images, it only gives me 200(the batch size) classification results.

Hi, thanks for providing this code. I am having issue on changing caffemodel of vgg_face provided from http://www.robots.ox.ac.uk/~vgg/software/vgg_face/ and having above error.

I changed the old caffe format to new caffe format (layers --> layer) with your README hint.
But having valueerror.

Thank you very much.

The tensorflow retrain.py example uses a frozen inception model that is entirely defined within the graphdef. By frozen, I mean they convert all the variables to constants in the graph so they get stored to the graphdef proto. This way everything is in one file.

Would it be possible to add this as an option to this lib?

So given a network that reshapes some dense activations to a set of a feature maps, convert.py throws out:

Error encountered: Unknown layer type encountered: Reshape

Can support for reshape be added easily?

Because I only want to get the tensorflow prototext.

I want to train the tensorflow prototext under the tensorflow framework, but convert.py tells me I should pass the caffemodel into it. I'm wondering if there can be the way input the caffe prototxt without caffemodel and get the tensorflow code only.

i've installed the latest caffe-master
i'm trying to convert the follow 2 file to tensorflow model
https://gist.githubusercontent.com/ksimonyan/211839e770f7b538e2d8/raw/0067c9b32f60362c74f4c445a080beed06b07eb3/VGG_ILSVRC_16_layers_deploy.prototxt
http://www.robots.ox.ac.uk/~vgg/software/very_deep/caffe/VGG_ILSVRC_16_layers.caffemodel

the first time i tried, convert.py raised the error "Error encountered: Unknown layer type encountered: 4"
it's because the layer type name defined kaffe\layer.py is not extactly the same with those referred by VGG_ILSVRC_16_layers_deploy.prototxt
i modified the capital layer name in VGG_ILSVRC_16_layers_deploy.prototxt, then ran convert.py again

this time it raised the error Enum type "caffe.V1LayerParameter.LayerType" has no value named Convolution, the following the full message:
(tf-0.8.0-gpu-py27)dl@dl123:~/mxnet-org-bai/tools/caffe-tensorflow-master$ python convert.py --caffemodel VGG_ILSVRC_16_layers.caffemodel --data-output-path tfout.data --code-output-path tfout.code VGG_ILSVRC_16_layers_deploy.prototxt-bak-new
I tensorflow/stream_executor/dso_loader.cc:105] successfully opened CUDA library libcublas.so locally
I tensorflow/stream_executor/dso_loader.cc:105] successfully opened CUDA library libcudnn.so locally
I tensorflow/stream_executor/dso_loader.cc:105] successfully opened CUDA library libcufft.so locally
I tensorflow/stream_executor/dso_loader.cc:105] successfully opened CUDA library libcuda.so.1 locally
I tensorflow/stream_executor/dso_loader.cc:105] successfully opened CUDA library libcurand.so locally

WARNING: PyCaffe not found!
Falling back to a pure protocol buffer implementation.
* Conversions will be drastically slower.
* This backend is UNTESTED!

Traceback (most recent call last):
File "convert.py", line 60, in
main()
File "convert.py", line 56, in main
args.phase)
File "convert.py", line 27, in convert
transformer = TensorFlowTransformer(def_path, caffemodel_path, phase=phase)
File "/home/dl/mxnet-org-bai/tools/caffe-tensorflow-master/kaffe/tensorflow/transformer.py", line 221, in init
self.load(def_path, data_path, phase)
File "/home/dl/mxnet-org-bai/tools/caffe-tensorflow-master/kaffe/tensorflow/transformer.py", line 227, in load
graph = GraphBuilder(def_path, phase).build()
File "/home/dl/mxnet-org-bai/tools/caffe-tensorflow-master/kaffe/graph.py", line 140, in init
self.load()
File "/home/dl/mxnet-org-bai/tools/caffe-tensorflow-master/kaffe/graph.py", line 146, in load
text_format.Merge(def_file.read(), self.params)
File "/home/dl/anaconda2/envs/tf-0.8.0-gpu-py27/lib/python2.7/site-packages/google/protobuf/text_format.py", line 309, in Merge
return MergeLines(text.split('\n'), message, allow_unknown_extension)
File "/home/dl/anaconda2/envs/tf-0.8.0-gpu-py27/lib/python2.7/site-packages/google/protobuf/text_format.py", line 346, in MergeLines
_ParseOrMerge(lines, message, True, allow_unknown_extension)
File "/home/dl/anaconda2/envs/tf-0.8.0-gpu-py27/lib/python2.7/site-packages/google/protobuf/text_format.py", line 371, in _ParseOrMerge
allow_unknown_extension)
File "/home/dl/anaconda2/envs/tf-0.8.0-gpu-py27/lib/python2.7/site-packages/google/protobuf/text_format.py", line 473, in _MergeField
allow_unknown_extension)
File "/home/dl/anaconda2/envs/tf-0.8.0-gpu-py27/lib/python2.7/site-packages/google/protobuf/text_format.py", line 493, in _MergeField
_MergeScalarField(tokenizer, message, field, allow_multiple_scalars)
File "/home/dl/anaconda2/envs/tf-0.8.0-gpu-py27/lib/python2.7/site-packages/google/protobuf/text_format.py", line 629, in _MergeScalarField
value = tokenizer.ConsumeEnum(field)
File "/home/dl/anaconda2/envs/tf-0.8.0-gpu-py27/lib/python2.7/site-packages/google/protobuf/text_format.py", line 951, in ConsumeEnum
raise self._ParseError(str(e))
google.protobuf.text_format.ParseError: 11:9 : Enum type "caffe.V1LayerParameter.LayerType" has no value named Convolution.

Hi,

I tried to convert the VGG model for the Places2 dataset (from here). But I get the following error:

(tensorflow)AIs-MacBook-Pro:caffe-tensorflow-master AI$ ./convert.py deploy.prototxt vgg16_places2.caffemodel placesnet.npy placesnet.py
WARNING: PyCaffe not found!
Falling back to protocol buffer implementation.
This may take a couple of minutes.
Traceback (most recent call last):
  File "./convert.py", line 33, in <module>
    main()
  File "./convert.py", line 17, in main
    transformer = TensorFlowTransformer(def_path, data_path)
  File "/Users/AI/caffe-tensorflow-master/kaffe/tensorflow/transformer.py", line 184, in __init__
    self.load(def_path, data_path)
  File "/Users/AI/caffe-tensorflow-master/kaffe/tensorflow/transformer.py", line 188, in load
    self.graph = GraphBuilder(def_path, data_path).build()
  File "/Users/AI/caffe-tensorflow-master/kaffe/core.py", line 308, in build
    DataInjector(self.def_path, self.data_path).inject(graph)
  File "/Users/AI/caffe-tensorflow-master/kaffe/core.py", line 118, in __init__
    self.load()
  File "/Users/AI/caffe-tensorflow-master/kaffe/core.py", line 127, in load
    self.load_using_pb()
  File "/Users/AI/caffe-tensorflow-master/kaffe/core.py", line 140, in load_using_pb
    self.params = [pair(layer) for layer in layers if layer.blobs]
  File "/Users/AI/caffe-tensorflow-master/kaffe/core.py", line 138, in <lambda>
    pair = lambda layer: (layer.name, self.transform_data(layer))
  File "/Users/AI/caffe-tensorflow-master/kaffe/core.py", line 149, in transform_data
    data = np.squeeze(np.array(blob.data, dtype=np.float32).reshape(c_o, c_i, h, w))
ValueError: total size of new array must be unchanged

This seems to be the same issue as here. Is there any workaround that I can use?

Thanks

I seem to get an import error when I try running convert.py:

Traceback (most recent call last):
File "convert.py", line 7, in
from kaffe import KaffeError, print_stderr
File "/home/sheng/workspace/caffe-tensorflow/kaffe/init.py", line 1, in
from .graph import GraphBuilder, NodeMapper
File "/home/sheng/workspace/caffe-tensorflow/kaffe/graph.py", line 4, in
from .data import DataInjector
ImportError: No module named data

I'm running Anaconda python 2.7, if that helps. Any help would be appreciated.

When I use your program on vgg16, I am having memory problems. Conversion works (after upgrading files), and I am able to reconstruct the graph in memory. However, when I attempt to do a forward-pass with a batch size of 32 images, I run out of memory. The error happens after "initialization" of variables. However, to my knowledge, tf.initialize_all_variables() doesn't actually initialize anything -- the next time you run something, initialization is triggered; so it makes sense why the error doesn't occur when I am constructing the graph in memory.

The out of memory error happens when I attempt to run something in a session. Within the first few convolutional layers of VGG16 is generally when things run out of memory. For instance, when I run it on a GTX 980 (4GB of ram) my GPU runs out of memory and in around the 3rd convolutional layer when I run it on a GTX Titian X (12GB of ram). Do you have any thoughts about why this might be the case? I am using the most recent caffe + tensorflow.

Here's the bash script im using to preprocess. Directly following this I am loading into tensorflow. I am running your scripts from inside another repository, so there might be slight differences.

Anyt thoughts?

#/bin/bash

CAFFE_PATH=~/caffe

wget http://www.robots.ox.ac.uk/~vgg/software/very_deep/caffe/VGG_ILSVRC_16_layers.caffemodel
wget https://gist.githubusercontent.com/ksimonyan/211839e770f7b538e2d8/raw/0067c9b32f60362c74f4c445a080beed06b07eb3/VGG_ILSVRC_16_layers_deploy.prototxt

$CAFFE_PATH/build/tools/upgrade_net_proto_text VGG_ILSVRC_16_layers_deploy.prototxt tmp.prototxt
mv tmp.prototxt VGG_ILSVRC_16_layers_deploy.prototxt

$CAFFE_PATH/build/tools/upgrade_net_proto_binary VGG_ILSVRC_16_layers.caffemodel tmp.caffemodel
mv tmp.caffemodel VGG_ILSVRC_16_layers.caffemodel

./caffe-tensorflow/convert.py VGG_ILSVRC_16_layers_deploy.prototxt VGG_ILSVRC_16_layers.caffemodel vgg.npy vgg.py
echo "import sys
sys.path.append('caffe-tensorflow')
" | cat - vgg.py > tmp.py

mv tmp.py vgg.py
rm VGG_ILSVRC_16_layers.caffemodel VGG_ILSVRC_16_layers_deploy.prototxt

I updated tensorflow, which updated protocol buffer to version 3.0.0a4. I can't import any model anymore.

To reproduce the bug, go in the caffe-tensorflow/caffe folder, and execute the following code:

python
Python 2.7.10 (default, Jul 13 2015, 12:05:58) 
[GCC 4.2.1 Compatible Apple LLVM 6.1.0 (clang-602.0.53)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from google import protobuf
>>> protobuf.__version__
'3.0.0a4'
>>> import caffepb
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "caffepb.py", line 28, in <module>
        type=None),
  File "/Users/olivier/pythonvenv/tf/lib/python2.7/site-packages/google/protobuf/descriptor.py", line 652, in __new__
    _message.Message._CheckCalledFromGeneratedFile()
TypeError: Descriptors should not be created directly, but only retrieved from their parent.
>>> exit()

One of the current limits seems to be that you cannot use variant batch size or image size in the network. This is caused by the reshape and get_shape functions in the Network class. Because you need to set the 1st dimension of the placeholder of input batch data to be None to use variant batch size, and set the 2nd and 3rd dimensions to be None to use variant image size.

In what cases do we need to reshape the output like this? I found it is not necessary at least for alexNet. If not, I can do some changes and make a push.

First, thanks for putting this together, it's great to see!

I just wanted to add a note that for future work, I've found the padding on convolution and maxpool layers the hardest thing to translate over, since both Caffe and TensorFlow have complex behaviors around the edges. I believe there are even some combinations in Caffe that can't currently be expressed in TensorFlow (since it only supports VALID and SAME).

Right now the translation always assumes SAME. If I get a chance I'll try to add in a warning during the translation process if the original network has other padding, and send a PR, but I wanted to add this as a comment in case anyone else runs into this issue.

Thanks again for this project.

Hi,
I'm trying to convert a model trained on MNIST in caffe. The conversion to tensorflow is raising an error. The conversion utility is missing the number of channels param for the first Convolution layer. Instead of reading the 4d array as (64, 1, 5, 5) for Conv1 layer, it only recognizes the params as a 3d array viz., (64, 5, 5). Can you pls help me.

Regards