This repository contains the code and instructions to use the trained CNN models described in [1] to extract features for Offline Handwritten Signatures. It also includes links to download extracted features from the GPDS, MCYT, CEDAR and Brazilian PUC-PR datasets.
[1] Hafemann, Luiz G., Robert Sabourin, and Luiz S. Oliveira. "Learning Features for Offline Handwritten Signature Verification using Deep Convolutional Neural Networks" http://dx.doi.org/10.1016/j.patcog.2017.05.012 (preprint)
Topics:
- Installation: How to set-up the dependencies / download the models to extract features from new signatures
- Usage: How to use this code as a feature extractor for signature images
- Using the features in Matlab: A script to facilitate processing multiple signatures and saving the features in matlab (.mat) format
- Datasets: Download extracted features (using the proposed models) for the GPDS, MCYT, CEDAR and Brazilian PUC-PR datasets (.mat files - do not require any pre-processing code)
The code is written in Python 2. We recommend using the Anaconda python distribution (link), and create a new environment using:
conda create -n sigver -y python=2
source activate sigver
The following libraries are required
- Scipy version 0.18
- Pillow version 3.0.0
- OpenCV
- Theano
- Lasagne
They can be installed by running the following commands:
conda install -y opencv "scipy=0.18.0" "pillow=3.0.0"
conda install -y jupyter notebook matplotlib # Optional, to run the example in jupyter notebook
pip install "Theano==0.9"
pip install https://github.com/Lasagne/Lasagne/archive/master.zip
We tested the code in Ubuntu 16.04. This code can be used with or without GPUs - to use a GPU with Theano, follow the instructions in this link. Note that Theano takes time to compile the model, so it is much faster to instantiate the model once and run forward propagation for many images (instead of calling many times a script that instantiates the model and run forward propagation for a single image).
- Clone (or download) this repository
- Download the pre-trained models from the project page
- Save / unzip the models in the "models" folder
Or simply run the following:
git clone https://github.com/luizgh/sigver_wiwd.git
cd sigver_wiwd/models
wget "https://storage.googleapis.com/luizgh-datasets/models/signet_models.zip"
unzip signet_models.zip
Run python example.py
. This script pre-process a signature, and compares the feature vector obtained by the model to the results obtained by the author. If the test fails, please check the versions of Scipy and Pillow. I noticed that different versions of these libraries produce slightly different results for the pre-processing steps.
The following code (from example.py) shows how to load, pre-process a signature, and extract features using one of the learned models:
from scipy.misc import imread
from preprocess.normalize import preprocess_signature
import signet
from cnn_model import CNNModel
canvas_size = (952, 1360) # Maximum signature size
# Load and pre-process the signature
original = imread('data/some_signature.png', flatten=1)
processed = preprocess_signature(original, canvas_size)
# Load the model
model_weight_path = 'models/signet.pkl'
model = CNNModel(signet, model_weight_path)
# Use the CNN to extract features
feature_vector = model.get_feature_vector(processed)
# Multiple images can be processed in a single forward pass using:
# feature_vectors = model.get_feature_vector_multiple(images)
For an interactive example, use jupyter notebook:
jupyter notebook
Look for the notebook "interactive_example.ipynb". You can also visualize it directly here
While the code requires python (with the libraries mentioned above) to extract features, it is possible to save the results in a matlab format. We included a script that process all signatures in a folder and save the results in matlab files (one .mat file for each signature).
Usage:
python process_folder.py <signatures_path> <save_path> <model_path> [canvas_size]
Example:
python process_folder.py signatures/ features/ models/signet.pkl
This will process all signatures in the "signatures" folder, using the SigNet model, and save one .mat file in the folder "features" for each signatures. Each file contains a single variable named "feature_vector" with the features extracted from the signature.
To faciliate further research, we are also making available the features extracted for each of the four datasets used in this work (GPDS, MCYT, CEDAR, Brazilian PUC-PR), using the models SigNet and SigNet-F (with lambda=0.95).
Dataset | SigNet | SigNet-F |
---|---|---|
GPDS | GPDS_signet | GPDS_signet_f |
MCYT | MCYT_signet | MCYT_signet_f |
CEDAR | CEDAR_signet | CEDAR_signet_f |
Brazilian PUC-PR* | brazilian_signet | brazilian_signet_f |
There are two files for each user: real_X.mat and forg_X.mat. The first contains a matrix of size N x 2048, containing the feature vectors of N genuine signatures from that user. The second contains a matrix of size M x 2048, containing the feature vectors of each of the M skilled forgeries made targetting the user.
* Note: for the brazilian PUC-PR dataset, the first 10 forgeries are "Simple forgeries", while the last 10 forgeries are "Skilled forgeries".
f = load('real_2.mat')
% f.features: [Nx2048 single]
from scipy.io import loadmat
features = loadmat('real_2.mat')['features']
# features: numpy array of shape (M, 2048)
If you use our code, please consider citing the following paper:
[1] Hafemann, Luiz G., Robert Sabourin, and Luiz S. Oliveira. "Learning Features for Offline Handwritten Signature Verification using Deep Convolutional Neural Networks" http://dx.doi.org/10.1016/j.patcog.2017.05.012 (preprint)
If using any of the four datasets mentioned above, please cite the paper that introduced the dataset:
GPDS: Vargas, J.F., M.A. Ferrer, C.M. Travieso, and J.B. Alonso. 2007. “Off-Line Handwritten Signature GPDS-960 Corpus.” In Document Analysis and Recognition, 9th I nternational Conference on, 2:764–68. doi:10.1109/ICDAR.2007.4377018.
MCYT: Ortega-Garcia, Javier, J. Fierrez-Aguilar, D. Simon, J. Gonzalez, M. Faundez-Zanuy, V. Espinosa, A. Satue, et al. 2003. “MCYT Baseline Corpus: A Bimodal Biometric Database.” IEE Proceedings-Vision, Image and Signal Processing 150 (6): 395–401.
CEDAR: Kalera, Meenakshi K., Sargur Srihari, and Aihua Xu. 2004. “Offline Signature Verification and Identification Using Distance Statistics.” International Journal of Pattern Recognition and Artificial Intelligence 18 (7): 1339–60. doi:10.1142/S0218001404003630.
Brazilian PUC-PR: Freitas, C., M. Morita, L. Oliveira, E. Justino, A. Yacoubi, E. Lethelier, F. Bortolozzi, and R. Sabourin. 2000. “Bases de Dados de Cheques Bancarios Brasilei ros.” In XXVI Conferencia Latinoamericana de Informatica.