This is the official Tensorflow implementation of the paper:
Yizhi Wang and Zhouhui Lian. Exploring Font-independent Features for Scene Text Recognition. ACM Multimedia. 2020.
As scene texts in novel font styles only make up a small proportion in existing benchmarks, we collect 100 text images with novel or unusual font styles to form a new dataset named as the Novel Font Scene Text (NFST) dataset (download link).
We compare our method with other two state-of-the-art methods (ASTER and SAR) whose codes are publicly available. Our method significantly outperforms others on this dataset (see the following table), whose robustness to font style variance is proved.
Training data | Ours | SAR | ASETR |
---|---|---|---|
90K+ST | 55% | 45% | 44% |
90K+ST+SA+R | 71% | 63% | 58% |
- python 3.6
- TensorFlow 1.15
Please use Anaconda to build the environment (otherwise, some errors may occur):
conda create -n efifstr python=3.6
source activate efifstr
pip install editdistance edit_distance
conda install tensorflow-gpu=1.15 pillow matplotlib
We reuse some code from ASTER, which is derived from Tensorflow Object Detection API.
- Go to
c_ops/
and runbuild.sh
to build the custom operators - Execute
protoc protos/*.proto --python_out=.
to build the protobuf files
The pretrained model files are available on this link (Update: new link has better performance). Download pretrained-model.zip
and extract it under experiments/demo/log/
.
To test the preformance of the pretrained-model, simply execute:
python eval.py
This will output the recognition result of NFSF dataset.
To test on a single image (data/demo.jpg), simply execute:
python demo.py
Data preparation scripts for several popular scene text datasets are located under tools
. See their source code for usage.
The multi-font glyph images are available on this link. Download glyphs-325-fonts.npy
and put it under data/
.
To run the example training, run
python train.py
Change the configuration in experiments/demo/trainval.prototxt
to configure your own training process.
During the training, you can run a separate program to repeatedly evaluates the produced checkpoints.
python eval.py
Evaluation configuration is also in trainval.prototxt
.