xiangrufan / keras-mtcnn

mtcnn face detection transplanted to keras

mtcnn-caffe

Keras Implementation of Joint Face Detection and Alignment using Multi-task Cascaded Convolutional Neural Networks.

This project provide you a method to update multi-task-loss for multi-input source.

Transplanted from MTCNN-caffe from CongweiLin's github repository github.com/CongWeilin/mtcnn-caffe

training requires Wider Face Training data set and CelebA data set (same as the caffe version).

However, the scripted is modified to reduce hard-disk usage. i.e. all intermediate cropped imgs are stored in memory. Requires at least 16 Gb memory to precess training data.

refined training process

The refined training uses training strategy that closely follows the original caffe code. i.e. randomly select Classification loss, roi regression loss or key point regression losses and minimize it for each batch of data. Slightly improve the performance. But also makes the training code unnecessarrily complex. Accuracy measurement is not implemented.

keras-mtcnn's Issues

ValueError: Error when checking target: expected classifier1 to have 4 dimensions, but got array with shape (39760, 2)

when i tried to uncomment the part you said
#classifier = Reshape((2,))(classifier) # this layer has to be deleted in order to enalbe arbitraty shape input
bbox_regress = Conv2D(4, (1, 1),name='bbox1')(x)
bbox_regress = Reshape((4,))(bbox_regress)
my_adam = adam(lr = 0.00001)

for i_train in range(80):
randx=random.choice([0,1,1]) # still need to run manually on each batch
# randx = 4
# randx = random.choice([ 4])
batch_size = 64
print ('currently in training macro cycle: ',i_train,'rand val:',randx)
if i_train ==0:
model = Model([input], [classifier, bbox_regress])
#model.load_weights('model12.h5',by_name=True)
bbox = model.get_layer('bbox1')
bbox_weight = bbox.get_weights()
classifier_dense = model.get_layer('classifier1')
cls_weight = classifier_dense.get_weights()

if 0 == randx:
    model = Model([input], [classifier])
    model.get_layer('classifier1').set_weights(cls_weight)
    model.compile(loss='mse', optimizer=my_adam, metrics=["accuracy"])
    model.fit(ims_cls, one_hot_labels, batch_size=batch_size, epochs=1)
    classifier_softmax = model.get_layer('classifier1')
    cls_weight = classifier_softmax.get_weights()

if 1 == randx:
    model = Model([input], [bbox_regress])
    model.get_layer('bbox1').set_weights(bbox_weight)
    model.compile(loss='mse', optimizer=my_adam, metrics=["accuracy"])
    model.fit(ims_roi, roi_score, batch_size=batch_size, epochs=1)
    bbox_dense = model.get_layer('bbox1')
    bbox_weight = bbox_dense.get_weights()

the error
Traceback (most recent call last):
File "keras_12net_v1.py", line 75, in
model.fit(ims_cls, one_hot_labels, batch_size=batch_size, epochs=1)
File "build/bdist.linux-x86_64/egg/keras/engine/training.py", line 1637, in fit
File "build/bdist.linux-x86_64/egg/keras/engine/training.py", line 1487, in _standardize_user_data
File "build/bdist.linux-x86_64/egg/keras/engine/training.py", line 113, in _standardize_input_data
ValueError: Error when checking target: expected classifier1 to have 4 dimensions, but got array with shape (39760, 2)

Are the 12net.h5 24net.h5 48net.h5 pretrained for MTCNN

Hi Xiangru,

Thanks a lot for the Keras example to train and use model. I tried to use the .h5 file to detect faces but looks like it fails to find any face... My question is that the whether the .h5 file you provide can be used to detect without retraining. If so I will go and find what I am missing.

Thanks

keras_ver_Onet_v7.py

python keras_ver_Onet_v7.py for bug:
OSError: Unable to open file (Unable to open file: name = 'onet7.h5', errno = 2, error message = 'no such file or directory', flags = 0, o_flags = 0)

Landmarks

How do I get landmarks as in this

ValueError: Dimension 0 in both shapes must be equal, but are 3 and 48 for 'Assign_9' (op: 'Assign') with input shapes: [3,3,28,48], [48,31,3,3].

$ python Run_model_caffe_weight.py
/usr/local/lib/python2.7/dist-packages/h5py/init.py:36: FutureWarning: Conversion of the second argument of issubdtype from float to np.floating is deprecated. In future, it will be treated as np.float64 == np.dtype(float).type.
from ._conv import register_converters as _register_converters
Using TensorFlow backend.
2018-03-01 00:34:39.126798: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.1 instructions, but these are available on your machine and could speed up CPU computations.
2018-03-01 00:34:39.126828: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.2 instructions, but these are available on your machine and could speed up CPU computations.
2018-03-01 00:34:39.126839: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX instructions, but these are available on your machine and could speed up CPU computations.
2018-03-01 00:34:39.126848: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX2 instructions, but these are available on your machine and could speed up CPU computations.
2018-03-01 00:34:39.126854: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use FMA instructions, but these are available on your machine and could speed up CPU computations.
2018-03-01 00:34:39.314734: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:893] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2018-03-01 00:34:39.315198: I tensorflow/core/common_runtime/gpu/gpu_device.cc:940] Found device 0 with properties:
name: GeForce 940MX
major: 5 minor: 0 memoryClockRate (GHz) 1.2415
pciBusID 0000:01:00.0
Total memory: 1.96GiB
Free memory: 1.68GiB
2018-03-01 00:34:39.315220: I tensorflow/core/common_runtime/gpu/gpu_device.cc:961] DMA: 0
2018-03-01 00:34:39.315228: I tensorflow/core/common_runtime/gpu/gpu_device.cc:971] 0: Y
2018-03-01 00:34:39.315236: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1030] Creating TensorFlow device (/gpu:0) -> (device: 0, name: GeForce 940MX, pci bus id: 0000:01:00.0)
Traceback (most recent call last):
File "Run_model_caffe_weight.py", line 11, in
Rnet = create_Kao_Rnet(r'24net.h5')
File "/home/akhil/Downloads/keras-mtcnn-master/Untitled Folder/MTCNN.py", line 53, in create_Kao_Rnet
model.load_weights(weight_path, by_name=True)
File "build/bdist.linux-x86_64/egg/keras/engine/topology.py", line 2649, in load_weights
File "build/bdist.linux-x86_64/egg/keras/engine/topology.py", line 3275, in load_weights_from_hdf5_group_by_name
File "build/bdist.linux-x86_64/egg/keras/backend/tensorflow_backend.py", line 2365, in batch_set_value
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/variables.py", line 516, in assign
return state_ops.assign(self._variable, value, use_locking=use_locking)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/state_ops.py", line 271, in assign
validate_shape=validate_shape)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/gen_state_ops.py", line 45, in assign
use_locking=use_locking, name=name)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/op_def_library.py", line 767, in apply_op
op_def=op_def)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/ops.py", line 2508, in create_op
set_shapes_for_outputs(ret)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/ops.py", line 1873, in set_shapes_for_outputs
shapes = shape_func(op)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/ops.py", line 1823, in call_with_requiring
return call_cpp_shape_fn(op, require_shape_fn=True)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/common_shapes.py", line 610, in call_cpp_shape_fn
debug_python_shape_fn, require_shape_fn)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/common_shapes.py", line 676, in _call_cpp_shape_fn_impl
raise ValueError(err.message)
ValueError: Dimension 0 in both shapes must be equal, but are 3 and 48 for 'Assign_9' (op: 'Assign') with input shapes: [3,3,28,48], [48,31,3,3]

p_net、r_net、o_net网络的H5文件训练

您好！我想修改p_net、r_net、o_net的网络结构，请问该如何训练每个网络的h5文件呢？我直接运行Keras_12net_v1.py文件结果程序不能运行。请问一下gen_celeba_data_lmdb.py , gen_net_data_lmdb.py , data_quality_confirmation.py 这几个程序的作用是使用celeba数据集生成人脸、非人脸和部分人脸的数据集的嘛？谢谢

Train on custom dataset

How do i train on a custom dataset?

How to save data into .imdb format file in "gen_net_data_lmdb.py"

Hi,

First I want to thank you for your work, which gives me a better understanding on MTCNN.

I am confused about the last several lines of the codes in "gen_net_data_lmdb.py" since I cannot find any information about the format ".imdb". Are you actually using the lmdb format since the name of this Python script used "lmdb"?

Hope to hear from you. Thanks.

Do the training codes work?

At first, great thanks to your hard work, I think it is a very good study material for a newbie like me.

BTW, I've tried to rebuild model files for each network (12, 24, 48) by running the codes you provided in 'training' folder.

I handled some minor mimatches between the training code and the network definitions in MTCNN.py, and managed to run training codes, and I now I have 3 model files. (with CelebA dataset only (gen_celebA_data_lmdb.py), do I have to include WIDER_FACE too? Is it mandatory?)

Unfortunately, if I replace old model files with my newly generated ones, MTCNN does not work at all, it fails to find ANY FACE.

How can I build my own model files?

Thanks in advance.

Cannot apply softmax to a tensor that is not 2D or 3D. Here, ndim=4

Hi, I try to python keras_12net_v1.py for bug:

ValueError: Cannot apply softmax to a tensor that is not 2D or 3D. Here, ndim=4

Thanks!

ValueError: No model found in config file.

---

ValueError Traceback (most recent call last)
in
----> 1 pnet=load_model("./12net.h5")
2 # rnet=load_model("./24net.h5")
3 # onet=load_model("./48net.h5")

/anaconda2/envs/python3/lib/python3.6/site-packages/tensorflow_core/python/keras/saving/save.py in load_model(filepath, custom_objects, compile)
144 if (h5py is not None and (
145 isinstance(filepath, h5py.File) or h5py.is_hdf5(filepath))):
--> 146 return hdf5_format.load_model_from_hdf5(filepath, custom_objects, compile)
147
148 if isinstance(filepath, six.string_types):

/anaconda2/envs/python3/lib/python3.6/site-packages/tensorflow_core/python/keras/saving/hdf5_format.py in load_model_from_hdf5(filepath, custom_objects, compile)
163 model_config = f.attrs.get('model_config')
164 if model_config is None:
--> 165 raise ValueError('No model found in config file.')
166 model_config = json.loads(model_config.decode('utf-8'))
167 model = model_config_lib.model_from_config(model_config,

ValueError: No model found in config file.

model12.h5、model24.h5和model48.h5如何训练？

您好！请问一下model12.h5、model24.h5和model48.h5是如何训练得到的？