BCALM 2 wtf: repart(max_minimizer) < repart(min_minimizer) about gatb-core

[rchikhi]

Hi Nick!
Input looks like what BCALM can take, and command line looks good. Variable-length reads are fine.

1. can you please pastebin the full output?
2. did bcalm work for you on another dataset, or does that bug occur for every dataset you try?

[NickSto]

Hi Ryan, thanks for the reply! Paul pointed me to this tool and it looks like it could really help me out of a bind.
I should've mentioned, I did try it on the example/tiny_read.fa dataset (using example/run-tiny.sh) and it finished fine, without this bug.
The full stdout is here, and the stderr is here.

[rchikhi]

I see nothing out of the ordinary in the log files, except maybe:

1. 48 threads is more threads than I generally test with. Please try with "-nb-cores 1" added as a command line parameter
2. if it doesn't work, would you mind sending me the file? I'm on brubeck.

[NickSto]

Oh, weird, I don't know how it ended up using 48 threads. I set it to 1 and it didn't fail with that error anymore! However, it failed with a different one:

[...]
Done with all compactions   --  memory [current, maximum (maxRSS)]: [ 661, 2177] MB 
Nb bglue threads: 1
unitigs graph destructor called
EXCEPTION: Unable to open bank 'k21a10n1/SC8C1.bcalm.unitigs.fa.glue'

And the output directory is no longer filled with dozens of files like SC8C1.bcalm.unitigs.fa.glue.44 and SC8C1.bcalm.unitigs.fa.doubledKmers.29, but instead with only SC8C1.bcalm.unitigs.fa.glue, SC8C1.bcalm.unitigs.fa.glue.0, and SC8C1.bcalm.h5.
The input file is in scratch5, at scratch5/nick/family/graph/bcalm/SC8C1/SC8C1.fq.gz. I'll also try with some different inputs, just to check.

[NickSto]

Update: I've narrowed it down to the choice of input.
I tried going back to the raw data, before quality filtering, and it worked! Somehow, the difference in the FASTQ files before and after filtering did it.
I filtered using sickle (v1.33), which trims bad ends off reads and filters them out if there's not enough good sequence left. Here's the specific sickle command I used:
$ sickle pe -t sanger -q 20 -l 100 -f $fastq1in -r $fastq2in -o $fastq1out -p $fastq2out -s $fastqs
You can find the raw data on brubeck at:
scratch1/boris/pnas_2014/mt.data/SC8C1-bl_[12].fq
and the filtered data at:
scratch5/nick/family/all4/fastq/filt/SC8C1-bl_[12].fq
Update to the update: And now I can't reproduce the case where it works. No matter how I manipulate the input, I keep getting EXCEPTION: Unable to open bank 'filename.bcalm.unitigs.fa.glue'.
But that bank file exists and contains the name of another file, filename.bcalm.unitigs.fa.glue.0, which contains valid FASTA sequences.

[rchikhi]

Thanks for the data. I can reproduce the issue now. I think it is due to the "-out" argument pointing to another folder, if you remove this argument or ask to output in the same folder, it doesn't crash. I'll work on a fix.

[NickSto]

Argh! Yep, that was it. Thanks so much for helping to figure this out!

[rchikhi]

you're welcome, and i'm a bit embarassed about that bug in such a basic functionality :) let me know if you stumble on other issues with bcalm. especially if the original "wtf traveller kmer" error resurfaces.

[rchikhi]

the -out bug has been fixed by commit 11a2aa1