ocr-d / ocrd_kraken

Wrapper for the kraken OCR engine

ocrd_kraken

OCR-D wrapper for the Kraken OCR engine

Introduction

This package offers OCR-D compliant workspace processors for (some of) the functionality of Kraken.

(Each processor is a parameterizable step in a configurable workflow of the OCR-D functional model. There are usually various alternative processor implementations for each step. Data is represented with METS and PAGE.)

It includes image preprocessing (binarization), layout analysis (region and line+baseline segmentation), and text recognition.

Installation

With Docker

This is the best option if you want to run the software in a container.

You need to have Docker

docker pull ocrd/kraken

To run with Docker:

docker run --rm \
-v path/to/workspaces:/data \
-v path/to/models:/usr/local/share/ocrd-resources \
ocrd/kraken ocrd-kraken-recognize ...
# or ocrd-kraken-segment or ocrd-kraken-binarize

Native, from PyPI

This is the best option if you want to use the stable, released version.

pip install ocrd_kraken

Native, from git

Use this option if you want to change the source code or install the latest, unpublished changes.

We strongly recommend to use venv.

git clone https://github.com/OCR-D/ocrd_kraken
cd ocrd_kraken
sudo make deps-ubuntu # or manually from git or via ocrd_all
make deps        # or pip install -r requirements.txt
make install     # or pip install .

Models

Kraken uses data-driven (neural) models for segmentation and recognition, but comes with no pretrained "official" models. There is a public repository of community-provided models, which can also be queried and downloaded from via kraken standalone CLI. (See Kraken docs for details.)

For the OCR-D wrapper, since all OCR-D processors must resolve file/data resources in a standardized way, there is a general mechanism for managing models, i.e. installing and using them by name. We currently manage our own list of recommended models (without delegating to the above repo).

Models always use the filename suffix .mlmodel, but are just loaded by their basename.

See the OCR-D model guide and

ocrd resmgr --help

Usage

For details, see docstrings in the individual processors and ocrd-tool.json descriptions, or simply --help.

Available OCR-D processors are:

• ocrd-kraken-binarize (nlbin – not recommended)
  • adds AlternativeImage files (per page, region or line) to the output fileGrp
• ocrd-kraken-segment (all-in-one segmentation – recommended for handwriting and simply layouted prints)
  • adds TextRegions, TableRegions, ImageRegions, MathsRegions, NoiseRegions, ReadingOrder and AlternativeImage to Page (depending on model training)
  • adds TextLines to TextRegions, including their Baseline
• ocrd-kraken-recognize (benefits from annotated Baselines, falls back to center-normalized bboxes)
  • adds Words to TextLines
  • adds Glyphs to Words
  • adds TextEquiv

Testing

make test

This downloads test data from https://github.com/OCR-D/assets under repo/assets, and runs some basic tests of the Python API.

Set PYTEST_ARGS="-s --verbose" to see log output (-s) and individual test results (--verbose).

ocrd_kraken's Issues

use multi-model recognition

Kraken offers "multi-script" (actually multi-model) prediction in one pass, so instead of a fixed model, we could run with multiple models and use the annotated language and script mappings to select per-segment (as in ocrd-tesserocr-recognize with xpath_model).

IIUC, that would entail using mm_rpred (instead of rpred) and passing lang/script to bounds['boxes'][...]['tags'] (or bounds['lines'][...]['tags'] with baseline segmentation) and a dict from lang/script to model names as the first arg.

Experiment with kraken's trainable reading order detection

Calling binarization twice will double file entry with identical ID in mets.xml

Before writing output to METS the script should test for ID.

Complete implementation of OCR with kraken

Text recognition is e.g. referenced in ocrd-tool.json but not in setup.py.

Rename "kraken-segment" to "kraken-segment-line"

It is in fact only line segmentation!

Default input_file_grp / output_file_grp

should be OCR-D-IMG and OCR-D-IMG-BIN-KRAKEN resp.

segmentation: add optional level=region (line segmentation only)

According to the OCR-D functional model, line segmentation takes place after block segmentation. It should therefor check for existing text blocks and restrict its area of operation to them.

Binarization will not adopt GROUPID from source.

The entry of the binarized image inside the METS file doesn't contain the GROUPID.

fallback to CPU if no GPU

It's unfortunate that Kraken itself requires selecting the computing device to have Pytorch use in advance.

For practical purposes, workflows should try to use CUDA if available. That's why ocrd_detectron2 falls back to cpu.

This should be implemented (and then documented) here as well.

Restrict segmentation to print space

If a print space is set via the corresponding element it has to be respected for (block) segmentation.

ocrd-tool invalid

@VolkerHartmann reports

ocrd validate-ocrd-tool ocrd-tool.json für KRAKEN liefert: ...
is not of type 'object'

kraken-binarize produces two identical output files per input file

When calling kraken binarization on a mets.xml like
ocrd process -m mets.xml kraken-binarize -I DEFAULT -O OCR-D-IMG-BIN
I obtain 2 identical binarized images per single input image, one residing in the CWD and another one in a directory OCR-D-IMG-BIN.

Possibly related to #10, #16?

Binarization creates 2 source files in target workspace

While binarizing image to a new workspace two tifs will be created.
The filename of the image is not the filename of the given mets.xml!
It seems the filename of the METS file in the cache directory!?

(source) Files are stored in the root directory of the workspace and looks like this:
file.path.to.old.workspace.filename
and
file.path.to.new.workspace.file.path.to.old.workspace.filename

The original file (OCR-D-IMG/filename) is missing in the new workspace!
(Inside METS is a reference to the first file mentioned above!)
Steps:

ocrd workspace validate
ocrd workspace clone -a -m mets.xml
cd /tmp/pyocrd-'xyz'
ocrd-kraken-binarize -w /new/target/dir

Pass on pageID

ocrd-kraken-binarize is not passing on the pageID

Binarization: Source image is copied to new workspace although it's already local

But mets.xml still reference the original image.
The filename looks strange 'file.xxx' and is not like the schema normally used .

ocrd-kraken-ocr process call seems to be broken

Traceback (most recent call last):
  File "/home/j23d/.local/share/virtualenvs/ocrd_butler-o_KhKE38/lib/python3.6/site-packages/celery/app/trace.py", line 385, in trace_task
    R = retval = fun(*args, **kwargs)
  File "/home/j23d/projects/ocrd_butler/ocrd_butler/celery_utils.py", line 21, in __call__
    return TaskBase.__call__(self, *args, **kwargs)
  File "/home/j23d/.local/share/virtualenvs/ocrd_butler-o_KhKE38/lib/python3.6/site-packages/celery/app/trace.py", line 648, in __protected_call__
    return self.run(*args, **kwargs)
  File "/home/j23d/projects/ocrd_butler/ocrd_butler/execution/tasks.py", line 82, in create_task
    **kwargs)
  File "/home/j23d/.local/share/virtualenvs/ocrd_butler-o_KhKE38/lib/python3.6/site-packages/ocrd/processor/base.py", line 56, in run_processor
    processor.process()
  File "/home/j23d/.local/share/virtualenvs/ocrd_butler-o_KhKE38/lib/python3.6/site-packages/ocrd_kraken/ocr.py", line 44, in process
    content=bin_image_bytes.getvalue())
  File "/home/j23d/.local/share/virtualenvs/ocrd_butler-o_KhKE38/lib/python3.6/site-packages/ocrd/workspace.py", line 162, in add_file
    raise Exception("'content' was set but no 'local_filename'")
Exception: 'content' was set but no 'local_filename'

I suspect that the call is not up to date with the current version of OCR-D core.

documentation: README completeness, debug ocrd-tool.json

Please debug your ocrd_tool.json file.
I found some errors:

<report valid="false">
  <error>[tools.ocrd-kraken-binarize.input_file_grp] 'OCR-D-IMG' is not of type 'array'</error>
  <error>[tools.ocrd-kraken-binarize.output_file_grp] 'OCR-D-IMG-BIN' is not of type 'array'</error>
  <error>[tools.ocrd-kraken-binarize.parameters.level-of-operation] 'description' is a required property</error>
  <error>[tools.ocrd-kraken-segment] 'input_file_grp' is a required property</error>
  <error>[tools.ocrd-kraken-segment] 'output_file_grp' is a required property</error>
  <error>[tools.ocrd-kraken-segment.parameters.maxcolseps] 'description' is a required property</error>
  <error>[tools.ocrd-kraken-segment.parameters.scale] 'description' is a required property</error>
  <error>[tools.ocrd-kraken-segment.parameters.black_colseps] 'description' is a required property</error>
  <error>[tools.ocrd-kraken-segment.parameters.white_colseps] 'description' is a required property</error>
  <error>[tools.ocrd-kraken-ocr] 'input_file_grp' is a required property</error>
  <error>[tools.ocrd-kraken-ocr] 'output_file_grp' is a required property</error>
  <error>[tools.ocrd-kraken-ocr.parameters.lines-json.required] 'true' is not of type 'boolean'</error>
</report>

You can find the ocrd-tool.json documentation: https://ocr-d.github.io/ocrd_tool

Please check your README file and complet them. An ideal README file look like:

# Name of application


## Introduction
...

## Installation
...

## Usage
...

## Testing
...

Thank you very much.

recognize: word coordinates are often invalid

Currently, the _make_word approach (creating a Word with dummy coordinates first, then adding points glyph by glyph

ocrd_kraken/ocrd_kraken/recognize.py

Line 99 in 802c6b0

current_word.get_Coords().points += ' ' + points_from_polygon(poly)

and finally them to a bbox when the next word starts,

ocrd_kraken/ocrd_kraken/recognize.py

Line 94 in 802c6b0

current_word.get_Coords().points = points_from_bbox(*bbox_from_polygon(polygon_from_points(current_word.get_Coords().points.strip())))

IIUC) creates polygons which are semantically unsound, e.g. 141,1263 141,1343 141,1343 141,1263 (notice the same points repeating, so we actually have only 2 instead of 4 here).

Support --version

It would be nice if all tools support printing version.
ocrd-kraken-binarize --version is not supported.

License template not completely filled out

The LICENSE does not contain the year and name of the copyright owner:

   Copyright [yyyy] [name of copyright owner]

Binarization doesn't respect output group

ocrd-kraken-segment creates negative coordinates (=invalid PAGE)

Hi,

I have an example, where ocrd-kraken-segment creates negative coordinates (=invalid PAGE).
I just have used:

ocrd resmgr download ocrd-kraken-segment blla.mlmodel
ocrd-kraken-segment -I <inputFileGrp> -O <outputFileGrp>

example.zip
As Result I can see:

<pc:TextRegion id="region_line_36">
            <pc:Coords points="3040,382 3040,-2 3219,-2 3219,382 3216,575 3037,569"/>

Binarization: Created files have no file:GROUPID

ocrd-kraken-binarize --version
Version 0.0.1, ocrd/core 0.3.1

ocrd workspace find --file-grp OCR-D-KRAKEN-BIN --output-field ID
OCR-D-KRAKEN-BIN_0001
OCR-D-KRAKEN-BIN_0002

ocrd workspace find --file-grp OCR-D-KRAKEN-BIN --output-field groupId
None
None