qurator-spk / ocrd_repair_inconsistencies

Automatically re-order lines, words and glyphs to become textually consistent with their parents.

Caution

This was a one-off script, useful to solve a specific problem. We do not maintain it anymore, but in case you want to use it, we appreciate an e-mail to [email protected] 🕸

ocrd_repair_inconsistencies

Automatically re-order lines, words and glyphs to become textually consistent with their parents.

Introduction

PAGE-XML elements with textual annotation are re-ordered by their centroid coordinates iff such re-ordering fixes the inconsistency between their appropriately concatenated TextEquiv texts with their parent's TextEquiv text.

If TextEquiv is missing, skip the respective elements.

Where available, respect the annotated visual order:

• For regions vs lines, sort in top-to-bottom fashion, unless another textLineOrder is annotated.
  (Both left-to-right and right-to-left will be skipped currently.)
• For lines vs words and words vs glyphs, sort in left-to-right fashion, unless another readingDirection is annotated.
  (Both top-to-bottom and bottom-to-top will be skipped currently.)

This processor does not affect ReadingOrder between regions, just the order of the XML elements below the region level, and only if not contradicting the annotated textLineOrder/readingDirection.

We wrote this as a one-shot script to fix some files. Use with caution.

Installation

(In your venv, run:)

make deps     # or pip install -r requirements.txt
make install  # or pip install .

Usage

Offers the following user interfaces:

OCR-D processor CLI ocrd-repair-inconsistencies

To be used with PageXML documents in an OCR-D annotation workflow.

Example

Use the following script to repair OCR-D-GT-PAGE annotation in workspaces, and then replace it with the output on success:

#!/bin/bash
set -e

tmp_fg=FIXED_$RANDOM

ocrd-repair-inconsistencies -I OCR-D-GT-PAGE -O $tmp_fg

for f in "$tmp_fg"/*; do
  g="OCR-D-GT-PAGE/OCR-D-GT-PAGE_${f#${tmp_fg}/${tmp_fg}_}"
  cp "$f" "$g"
done

ocrd workspace remove-group -rf $tmp_fg

ocrd_repair_inconsistencies's Issues

Find a better name

It repairs inconsistencies in segment texts vs their children texts vs XML order and the name should reflect that.

generalize to other textLineOrder

Currently the repair does not apply when the region's @textLineOrder is something other than top-to-bottom. But the code could easily be generalized to all allowed values (while still assuming top-to-bottom when nothing is annotated).

Moreover, IMO readingDirection should not be checked at all on the region vs line level (only on the line vs word and word vs glyph levels).

Add license

add tests and CI

I imagine this can fail in many ways. Do you have good example data? Or rather, create them artificially by re-ordering segments in good GT ad-hoc?

As for negative tests, we could probably use kant_aufklaerung_1784 from OCR-D/assets because of its bad tokenization, plus some bags/filegrps without text or with missing text.

Integrate @bertsky's tokenization repair?

@bertsky wrote in #4

So, @mikegerber do you want me to integrate my tokenization repair solution here as well? (Perhaps to be activated by an extra parameter?)

Move to ocrd-segment-repair?

@bertsky wrote in #1:

I still think this would make a very good addition to ocrd-segment-repair...