zeiwoos / konfuzio-sdk

OCR, extract and classify documents. In addition, annotate documents and build your own NLP and Computer Vision models using Python by downloading the data. Find examples in our Colab Notebooks, e. g. how to fine-tune Flair.

Konfuzio SDK

The Konfuzio Software Development Kit (Konfuzio SDK) provides a Python API to interact with the Konfuzio Server.

Features

The SDK allows you to retrieve visual and text features to build your own document models. Konfuzio Server serves as an UI to define the data structure, manage training/test data and to deploy your models as API.

Function	Public Host Free*	On-Site (Paid)
OCR Text	✔️	✔️
OCR Handwriting	✔️	✔️
Text Annotation	✔️	✔️
PDF Annotation	✔️	✔️
Image Annotation	✔️	✔️
Table Annotation	✔️	✔️
Download Images	✔️	✔️
Download PDF with OCR	✔️	✔️
Deploy AI models	✖️	✔️

* Under fair use policy: We will impose 10 pages/hour throttling eventually.

📒 Docs	Read the docs
💾 Installation	How to install the Konfuzio SDK
🎓 Tutorials	See what the Konfuzio SDK can do with our Notebooks & Scripts
💡 Explanations	Here are links to teaching material about the Konfuzio SDK.
⚙️ API Reference	Python classes, methods, and functions
❤️ Contributing	Learn how to contribute!
🐛 Issue Tracker	Report and monitor Konfuzio SDK issues
🔭 Changelog	Review the release notes
📰 MIT License	Review the license

Installation

As developer register on our public HOST for free: https://app.konfuzio.com

Then you can use pip to install Konfuzio SDK and run init:

pip install konfuzio_sdk

konfuzio_sdk init

The init will create a Token to connect to the Konfuzio Server. This will create variables KONFUZIO_USER, KONFUZIO_TOKEN and KONFUZIO_HOST in an .env file in your working directory.

By default, the SDK is installed without the AI-related dependencies like torch or transformers and allows for using only the Data-related SDK concepts but not the AI models. To install the SDK with the AI components, run the following command:

pip install konfuzio_sdk[ai]

Find the full installation guide here or setup PyCharm as described here.

CLI

We provide the basic function to create a new Project via CLI:

konfuzio_sdk create_project YOUR_PROJECT_NAME

You will see "Project {YOUR_PROJECT_NAME} (ID {YOUR_PROJECT_ID}) was created successfully!" printed.

And download any project via the id:

konfuzio_sdk export_project YOUR_PROJECT_ID

Tutorials

You can find detailed examples about how to set up and run document AI pipelines in our Tutorials, including:

• Split a file into separate Documents
• Document Categorization
• Train a Konfuzio SDK Model to Extract Information From Payslip Documents

Basics

Here we show how to use the Konfuzio SDK to retrieve data hosted on a Konfuzio Server instance.

from konfuzio_sdk.data import Project, Document

# Initialize the Project
YOUR_PROJECT_ID: int
my_project = Project(id_=YOUR_PROJECT_ID)

# Get any Document online
DOCUMENT_ID_ONLINE: int
doc: Document = my_project.get_document_by_id(DOCUMENT_ID_ONLINE)

# Get the Annotations in a Document
doc.annotations()

# Filter Annotations by Label
MY_OWN_LABEL_NAME: str
label = my_project.get_label_by_name(MY_OWN_LABEL_NAME)
doc.annotations(label=label)

# Or get all Annotations that belong to one Category
YOUR_CATEGORY_ID: int
category = my_project.get_category_by_id(YOUR_CATEGORY_ID)
label.annotations(categories=[category])

# Force a Project update. To save time Documents will only be updated if they have changed.
my_project.get(update=True)