nlnwa / hermetic

Send and verify data to digitial storage

Hermetic

Sends data to digital storage

Usage

Send

hermetic send \
    --kafka-endpoints=<list-of-kafka-endpoints> \
    --transfer-topic <topic-name> \
    --stage-artifacts-root <stage-artifacts-root>

Verify

Reject

hermetic verify \
    --kafka-endpoints=<list-of-kafka-endpoints> \
    reject
      --reject-topic <topic-name>

Confirm

hermetic verify \
--kafka-endpoints=<list-of-kafka-endpoints> \
confirm
--confirm-topic <topic-name>

Acquisition upload

hermetic acquistion \
    --kafka-endpoints=<list-of-kafka-endpoints> \
    --transfer-topic <topic-name> \
    --acquisition-root=</path/to/root/of/acquisition>

hermetic's Issues

Use a logging framwork instead of print statements.

Motivation

As the application grows in complexity, the amount of print statements for logging purposes has increased. While print statements serve their purpose, they lack the structured approach necessary for effective debugging and monitoring.
To address this, we propose integrating a robust logging framework into our codebase.

Feature

We propose implementing structured logging using the slog package in Golang.
Integrating slog into our application will enable us to maintain better visibility into its behavior, streamline debugging processes, and facilitate monitoring and analysis.

Generate METS/PREMIS files for acquisitions

Motivation

When uploading an acquisition the future needs to know where exactly the files came from, who edited them, who owns the rights to this work, etc.
The current way of tracking this information is by using a custom YAML file, which is only a temporary solution.

Feature

By generating Metadata Encoding and Transmission StandardMETS and Preservation Metadata: Implementation StrategiesPREMIS files this problem could be solved.

Log `confirm` topic messages

Motivation

The current design proposal only validates the entries in the reject topic, but the messages sent to the confirm topic might still be of interest.

Feature

Logging at least the relevant results from the confirm topic could be helpful when debugging.