kuhnuri / kuhnuri-worker

Kuhnuri worker

Kuhnuri Worker

DITA-OT Worker that communicates with a Queue over REST.

Process

The Worker has a single loop where

1. Worker polls Queue for work
2. Queue returns a job
3. Worker downloads the source
4. Worker processes job
5. Worker uploads the results
6. Worker submits the results to Queue

If Queue has no work available, the process is

1. Worker polls Queue for work
2. Queue returns a no work
3. Worker takes a nap

Source and results support

The following URI schemes are supported for source content:

• file
• jar

The following URI schemes are supported for result content:

• file
• jar

Building

Compile the code:

1. sbt compile

Develoment

Running a development version:

1. sbt run

Running in Docker

The following volumes are used:

• /var/log/app: Logs directory
• /tmp/app: Temporary file directory
• /var/lib/app: State file directory

Deploying

Build a distribution package:

1. sbt dist

kuhnuri-worker's Issues

Support conversion cancel

Add support for cancelling conversion job while it's running.

Persist Worker job to disk

Save Worker job to disk so the job can be rerun after a failure and restart.

Read default transtypes list from plugins configuration

Read all available transtypes from plugins.xml and validate that worker.transtypes value is valid.

Retry submitting work

If submitting results fails, retry e.g. 3 times with 5 second intervals, and only then give up.

Restart conversion after crash

When Worker process crashes and is restarted, restart the conversion based on persisted conversion state.

Depends on:

• #1 Persist Worker job to disk

Use

• -XX:+ExitOnOutOfMemoryError
• -XX:ExitOnOutOfMemoryErrorExitCode=51

To make JVM signal out of memory error.

Worker that forks JVM for DITA-OT

Add a Worker implementation that forks JVM to run DITA-OT. This allows ad hoc DITA-OT use without needing to launch separate Worker for each DITA-OT.

Worker push mode

Processing mode where queue pushes worker jobs and worker either accepts the work or rejects it.

Bundle DITA-OT with distribution

Bundle DITA-OT with distribution build.

Add password support to queue registration

When registering to queue, add support for providing password.

Detect queue unavailability and retry to register

On startup, retry registering to queue until success.

If during processing queue becomes unavailable, retry registering.

Support older DITA-OT versions

Support DITA-OT versions older than 2.5. Older version do not have the Java API used to run DITA-OT from Worker.