adoni5 / mappy-rs

Python wrapping multithreaded bindings to minimap2

About me

Hi! I'm currently (10/11/2022) a post-graduate researcher at the University of Nottingham working on adaptive sampling with nanopore sequencing. I'm interested in all sorts of Bioinformatics, visualisations and fun side projects ( Some of which I even complete ).

Current mood

Current Projects

🐱‍💻 I'm currently working on:

minoTour - real-time monitoring of nanopore sequencers

Swordfish - An inter-communicator for minoTour and readfish

Advent Of Code 🎅🎅 - Just gotta love it

🔭 How to reach me:

Follow me on twitter for utter silence @rorymatics ◦ Follow me on mastadon [email protected] ◦ Open issues! ◦ 📧 [email protected]

mappy-rs's Issues

Add Scheduled.yml from Jon Gjengset rust ci suite

Scheduled jobs for every sunday - checks things like dependencies etc.
https://github.com/jonhoo/rust-ci-conf

Mans got a memory leak?

Threadbuffer potentially not freeing up memory.

• Profile with bytehound
• Implement freeing of threadbuffer as PR to minimap2-rs

Change Alignment fields to match Mappy

Seg faults if index doesn't exist

I guess add a pathbuf check?

Fix mappy interface

Require the addition of the __bool__ function and fix how the scoring is set on mapopts

Add safety.yml from Jon Gjengset rust github

Covers unsafe code, looking for unsanitised addresses and memory leaks

Run tests on multiple architectures

See for example:
https://github.com/messense/py-dissimilar/blob/d1d813ad287aab81a3730777021d61321503ce01/.github/workflows/CI.yml#L183

At the moment, I only know it compiles.

Cron Ci jobs

Once a week, run the two CI jobs

Send_one functionality

Send a read to be aligned into the work queue.
Needs a companion map function to map all the reads in the queue, yielding the results and the metadata. In this case, Metadata should consist of either channel_number, read_number or read_id, and seq.

Should be quite straight forward. 🥹

Allow map batch to accept lists

It would be nice if Aligner.map_batch could accept a list of input, currently:

import mappy_rs
fasta = [
  {'id': 0, 'seq': 'AGAGTGAAGCCAATATTCCGATAACGATTGCTTTCATGATATCCCTCATTCATCACAAGTTTT'},
  {'id': 1, 'seq': 'AGAGTGAAGCCAATATTCCGATAACGATTGCTTTCATGATATCCCTCATTCATCACAAGTTTT'},
]
al = mappy_rs.Aligner("resources/test/test.mmi")
al.enable_threading(2)
mappings = al.map_batch(fasta)
# TypeError: argument 'seqs': 'list' object cannot be converted to 'Iterator'

Fix Cigar functionality

Currently doesn't work!

Get reference seq out of .mmi files functionality

Badges in README

I need the Serotonin

Fix tests

The tests/test_alingment.yml is very out of date - update to use new format for system tests.
Don't
We also need to create more rust based unit tests in the lib.
For when I'm feeling bored!

Apple Silicon issues

Compiling on Apple Silicon seems impossible.

Error messages are:

error occurred: Command "cc" "-O2" "-ffunction-sections" "-fdata-sections" "-fPIC" "-gdwarf-2" "-fno-omit-frame-pointer" "-arch" "arm64" "-static" "-I" "minimap2" "-Wc++-compat" "-DHAVE_KALLOC" "-lm" "-lpthread" "-o" "/Users/mattloose/GIT/mappy-rs/target/debug/build/minimap2-sys-a801728ffb604384/out/minimap2/ksw2_ll_sse.o" "-c" "minimap2/ksw2_ll_sse.c" with args "cc" did not execute successfully (status code exit status: 1).

warning: build failed, waiting for other jobs to finish...

benchmarking

Benchmarking scripts could be placed in resources/benchmarking

Move from unmaintained `actions-rs/toolchain` to ` dtolnay/rust-toolchain`

https://github.com/dtolnay/rust-toolchain

action-rs/toolchain is about to be deprecated due to the move to node 16 only actions.

Documentation

Multi threaded interface

We can extend the base mappy-rs.Aligner, which is currently single-threaded, to use multiple threads on iterables of data.

Proposed minimal interface:

• mappy-rs.Aligner.send(data: PyDict): send a python dictionary containing at least one key/value pair, seq -> FASTA (string) (?). This function should return whether the data was queued successfully. Maybe take a second parameter that is the key to the FASTA data
• mappy-rs.Aligner.get_results: retrieve all available aligned data from the output queue and return it.

Extended interface:

• mappy-rs.Aligner.send_batch(batch: Iter[data, ...]): Place an iterable of data dictionaries into the work queue. These would be retrieved by get_results. This should be non-blocking.
• mappy-rs.Aligner.map_batch(batch: Iter[data, ...]): Map a batch of data, yielding results

Copy mappy python interface for drop-in compatibility

minimap2 already has a python interface, mappy we should aim to re-implement that API so that - in the first instance - this can function as a drop-in replacement.

For this we need:

• Same Aligner args
  • Take fn_idx_in as (MMI)
  • Take fn_idx_in as (FA)
  • Write to fn_idx_out (FA -> MMI)
• Same Aligner.map args
• Aligner.seq_names to work
• Aligner.seq to have the same behaviour
  • Return None on error rather than raise
• Same Alignment structure with attributes

Maybes:

• Misc functions:
  • fastx_read(fn, read_comment=False)
  • revcomp(seq)

Pre-commit

At some point I would like to set up pre-commit and rust-fmt and cargo test on hooks

__repr__ functions

Write __str__ and __repr__ functions for the Alignments.
__str__ should return a paf formatted line.