shadowjonathan / conduit_toolbox

🧰🦀 A bunch of tools that're helpful for conduit

🧰 Conduit Toolbox 🦀

Helper tools for Conduit, a matrix homeserver in rust.

This repository provides binaries to wrangle your conduit homeserver with.

Tools

conduit_sled_to_sqlite

This tool is no longer available on the main branch, go to with-sled to download from there.

conduit_migrate

This tool provides generic migration between heed, sqlite, persy, and rocksdb conduit databases.

Installing

For the best experience, compile this toolbox locally on your server;

1. Download and install rust, see rustup for more.
2. Be sure that the rust executables are on your $PATH
3. You may want to have a compiler and build tools installed on your system, or else cargo will complain about not being able to "link" or "compile" with cc.
   • on debian/ubuntu-based systems you can install this with sudo apt install build-essential
4. cargo install --locked --git https://github.com/shadowjonathan/conduit_toolbox conduit_migrate

(updating only requires running that last line again)

conduit_toolbox's Issues

Ability to convert synapse's signing key to DER PKCS#8 v2

Add persy backend

Add persy backend, according to the interface from this MR

Deprecate conduit_sled_to_sqlite

In favour of conduit_migrate

• Add banner to README
• Add alt instructions
• Remove actual binary target

Add flag to migration to `mv` media subdirectory

Change rocksdb config

From timo:

I changed the rocksdb config, you might need to copy that one over

Installation method does not work

The installation method suggested in README.md does not work:

❯ cargo install --locked --git https://github.com/shadowjonathan/conduit_toolbox
    Updating git repository `https://github.com/shadowjonathan/conduit_toolbox`
error: multiple packages with binaries found: conduit_migrate, conduit_sled_to_sqlite

Ability to extract the signing key from conduit

globals > keypair

RocksDB fixup tool

@timokoesters:

Okay short summary for you: rocksdb has a bug where it always sets a wrong compressions option if a specific rust feature is enabled, but we want to use another, incompatible, compression option. That means all people currently using rocksdb want to migrate from "rocksdb with feature A" to "rocksdb with feature B"

I'd rather want to have a fixup tool for this, so someone can run it "to just fix it"

Update README to include conduit_migrate

Unable to migrate from rocks to sqlite

I was trying to use this tool to move my conduit database from rocksdb to sqlite and it does not work correctly. I get the following error:

% conduit_migrate -f rocks -t sqlite -s conduit -d exit
[tools/migrate/src/main.rs:164:5] &src_dir = "/home/user/conduit"
[tools/migrate/src/main.rs:164:5] &dst_dir = "/home/user/exit"
Error: Corruption: unknown checksum type 4 from footer of /home/user/conduit/006562.sst, while checking block at offset 1035 size 79

It seems as if there is a problem with that particular file, however conduit does not complain about any database problems.