Nothing works! Please fix. @snurk

Hello,
I am running metaspades on a large soil metagenome, which I have partitioned into "high-", "low" and "medium"-coverage subsets prior to assembly using bbnorm (based on median kmer-counts for each read). I am assembling these fractions individually to reduce RAM-requirements.

For the assembly I am using a k-mer range from 21-99, with extra small k-mer steps towards the end, in order to ensure that i minimize k-mer gaps for my (low-abundant) target genomes (last k-mer steps=91,95,97,99 for ~101 bp HiSeq reads)

Assembly worked fine for the "low"- and "high"-abundant fractions.
It also seemed to work fine for most of the k-mer steps of the "medium"-abundant fraction. However during the forelast k-mer step (97), the "medium"-assembly keeps aborting with the following error message:

[...]  
  1:24:10.248     4G / 18G   INFO   K-mer Index Building     (kmer_index_builder.hpp    : 437)   Building perfect hash indices
  1:26:45.297     4G / 18G   INFO    General                 (kmer_index_builder.hpp    : 276)   Merging final buckets.
  1:27:12.929     4G / 18G   INFO   K-mer Index Building     (kmer_index_builder.hpp    : 483)   Index built. Total 52155392 bytes occupied (2.61376 bits per kmer).
  1:27:15.200     8G / 18G   INFO    General                 (edge_index_builders.hpp   :  27)   Collecting k-mer coverage information from graph, this takes a while.
  1:27:56.406     8G / 18G   INFO    General                 (edge_index.hpp            :  91)   Index refilled
  1:27:56.411     8G / 18G   INFO    General                 (gap_closer.cpp            : 159)   Preparing shift maps
  1:28:00.765     8G / 18G   INFO    General                 (gap_closer.cpp            : 119)   Processing paired reads (takes a while)
  1:29:45.203     8G / 18G   INFO    General                 (gap_closer.cpp            : 138)   Used 17384814 paired reads
  1:29:45.204     8G / 18G   INFO    General                 (gap_closer.cpp            : 140)   Merging paired indices
  1:29:49.090     8G / 18G   INFO   GapCloser                (gap_closer.cpp            : 349)   Closing short gaps
=== Stack Trace ===
[0x40d753]
[0x44d309]
[0x45942e]
[0x459ba3]
[0x4617dd]
[0x436358]
[0x546e21]
[0x40a328]
[0x4028ea]
[0x79acf0]
[0x4086fd]
spades: /spades/src/projects/spades/gap_closer.cpp:294: bool debruijn_graph::GapCloser::HandleSimpleCase(debruijn_graph::GapCloser::EdgeId, debruijn_graph::GapCloser::EdgeId, int): Assertion `overlap <= k_' failed.

== Error ==  system call for: "['/work/home/jov14/tools/SPAdes-3.10.0/bin/spades', '/work/home/jov14/tmp/projects/pratscher_USCa_metagenome2/results/assembly/170124_metaspades_bbmapmidpartition_alldatasets_k21-101/assembly_metaspades3.10/K97/configs/config.info', '/work/home/jov14/tmp/projects/pratscher_USCa_metagenome2/results/assembly/170124_metaspades_bbmapmidpartition_alldatasets_k21-101/assembly_metaspades3.10/K97/configs/mda_mode.info', '/work/home/jov14/tmp/projects/pratscher_USCa_metagenome2/results/assembly/170124_metaspades_bbmapmidpartition_alldatasets_k21-101/assembly_metaspades3.10/K97/configs/meta_mode.info']" finished abnormally, err code: -6

I have attached the params.txt and spades.log below.

Do you know what is causing this error? Will it be possible to continue this assembly, without restricting myself to a lower maximum-kmer (I want all of the fractions to be assembled using the same parameters).

params.txt

spades.log.txt

Hello,

I have some troubles trying to compile SPAdes-3.11.0 downloaded from home page: http://cab.spbu.ru/files/release3.11.0/SPAdes-3.11.0.tar.gz

see:

bigmess:/tmp/build > tar xf ~src/SPAdes/SPAdes-3.11.0.tar.gz
bigmess:/tmp/build > cd SPAdes-3.11.0 
bigmess:build/SPAdes-3.11.0 > ./spades_compile.sh -DCMAKE_VERBOSE_MAKEFILE=ON 2>&1 | tee log.txt


               [SNIP see attached log.tx for whole details]
[ 68%] Built target assembly_graph
Linking CXX static library libstages.a
cd /tmp/build/SPAdes-3.11.0/build_spades/common/stages && /mount/gensoft2/exe/cmake/3.2.1/bin/cmake -P CMakeFiles/stages.dir/cmake_clean_target.cmake
cd /tmp/build/SPAdes-3.11.0/build_spades/common/stages && /mount/gensoft2/exe/cmake/3.2.1/bin/cmake -E cmake_link_script CMakeFiles/stages.dir/link.txt --verbose=1
/usr/bin/ar cq libstages.a  CMakeFiles/stages.dir/construction.cpp.o CMakeFiles/stages.dir/simplification.cpp.o
/usr/bin/ranlib libstages.a
make[2]: Leaving directory `/tmp/build/SPAdes-3.11.0/build_spades'
/mount/gensoft2/exe/cmake/3.2.1/bin/cmake -E cmake_progress_report /tmp/build/SPAdes-3.11.0/build_spades/CMakeFiles  98
[ 68%] Built target stages
make[1]: Leaving directory `/tmp/build/SPAdes-3.11.0/build_spades'
make: *** [all] Error 2

I'm abble to compile this version using boost/1.61.0 headers, by doing the following.

bigmess:build/SPAdes-3.11.0 > cd ext/include
bigmess:ext/include > mv boost boost.no
bigmess:ext/include > ln -s /local/gensoft2/lib/boost/1_61_0/include/boost .
bigmess:ext/include > cd ../../
bigmess:build/SPAdes-3.11.0 > ./spades_compile.sh -DCMAKE_VERBOSE_MAKEFILE=ON                    

best regards

Eric

log.txt

The cov-cutoff parameter remains a mystery to the Spades user community. It used to be auto and now it is off.

Would it be possible to add an explaination to the document explaining it?

Common results are getting contigs with coverages of < 1.0

Is this a known issue or is it just my own system?

Command line: /usr/local/bin/spades.py --meta --pe1-1 /Users/mars/Desktop/spades_single_end/fastq/R1.fastq --pe1-2 /Users/mars/Desktop/spades_single_end/fastq/R2.fastq -o /Users/mars/Desktop/spades_single_end/fastq/metaspades_output

System information:
SPAdes version: 3.10.1
Python version: 2.7.13
OS: Darwin-16.4.0-x86_64-i386-64bit

Output dir: /Users/mars/Desktop/spades_single_end/fastq/metaspades_output
Mode: read error correction and assembling
Debug mode is turned OFF

Dataset parameters:
Metagenomic mode
Reads:
Library number: 1, library type: paired-end
orientation: fr
left reads: ['/Users/mars/Desktop/spades_single_end/fastq/R1.fastq']
right reads: ['/Users/mars/Desktop/spades_single_end/fastq/R2.fastq']
interlaced reads: not specified
single reads: not specified
Read error correction parameters:
Iterations: 1
PHRED offset will be auto-detected
Corrected reads will be compressed (with gzip)
Assembly parameters:
k: [21, 33, 55]
Repeat resolution is enabled
Mismatch careful mode is turned OFF
MismatchCorrector will be SKIPPED
Coverage cutoff is turned OFF
Other parameters:
Dir for temp files: /Users/mars/Desktop/spades_single_end/fastq/metaspades_output/tmp
Threads: 16
Memory limit (in Gb): 250

Command line: /usr/local/bin/spades.py --meta --pe1-1 /Users/mars/Desktop/spades_single_end/fastq/R1.fastq --pe1-2 /Users/mars/Desktop/spades_single_end/fastq/R2.fastq -o /Users/mars/Desktop/spades_single_end/fastq/metaspades_output

System information:
SPAdes version: 3.10.1
Python version: 2.7.13
OS: Darwin-16.4.0-x86_64-i386-64bit

Output dir: /Users/mars/Desktop/spades_single_end/fastq/metaspades_output
Mode: read error correction and assembling
Debug mode is turned OFF

Dataset parameters:
Metagenomic mode
Reads:
Library number: 1, library type: paired-end
orientation: fr
left reads: ['/Users/mars/Desktop/spades_single_end/fastq/R1.fastq']
right reads: ['/Users/mars/Desktop/spades_single_end/fastq/R2.fastq']
interlaced reads: not specified
single reads: not specified
Read error correction parameters:
Iterations: 1
PHRED offset will be auto-detected
Corrected reads will be compressed (with gzip)
Assembly parameters:
k: [21, 33, 55]
Repeat resolution is enabled
Mismatch careful mode is turned OFF
MismatchCorrector will be SKIPPED
Coverage cutoff is turned OFF
Other parameters:
Dir for temp files: /Users/mars/Desktop/spades_single_end/fastq/metaspades_output/tmp
Threads: 16
Memory limit (in Gb): 250

======= SPAdes pipeline started. Log can be found here: /Users/mars/Desktop/spades_single_end/fastq/metaspades_output/spades.log

===== Read error correction started.

== Running read error correction tool: /usr/local/Cellar/spades/3.10.1/bin/hammer /Users/mars/Desktop/spades_single_end/fastq/metaspades_output/corrected/configs/config.info

0:00:00.000 4M / 4M INFO General (main.cpp : 83) Starting BayesHammer, built from N/A, git revision N/A
0:00:00.000 4M / 4M INFO General (main.cpp : 84) Loading config from /Users/mars/Desktop/spades_single_end/fastq/metaspades_output/corrected/configs/config.info
0:00:00.001 4M / 4M INFO General (memory_limit.hpp : 47) Memory limit set to 250 Gb
0:00:00.001 4M / 4M INFO General (main.cpp : 93) Trying to determine PHRED offset
0:00:00.002 4M / 4M INFO General (main.cpp : 99) Determined value is 33
0:00:00.002 4M / 4M INFO General (hammer_tools.cpp : 36) Hamming graph threshold tau=1, k=21, subkmer positions = [ 0 10 ]
0:00:00.002 4M / 4M INFO General (main.cpp : 120) Size of aux. kmer data 24 bytes
=== ITERATION 0 begins ===
0:00:00.002 4M / 4M INFO K-mer Counting (kmer_data.cpp : 282) Estimating k-mer count
0:00:00.064 132M / 132M INFO K-mer Counting (kmer_data.cpp : 287) Processing /Users/mars/Desktop/spades_single_end/fastq/R1.fastq
0:00:52.307 160M / 160M INFO K-mer Counting (kmer_data.cpp : 296) Processed 17262970 reads
0:00:52.307 160M / 160M INFO K-mer Counting (kmer_data.cpp : 287) Processing /Users/mars/Desktop/spades_single_end/fastq/R2.fastq
0:01:53.443 160M / 160M INFO K-mer Counting (kmer_data.cpp : 296) Processed 34525940 reads
0:01:53.443 160M / 160M INFO K-mer Counting (kmer_data.cpp : 301) Total 34525940 reads processed
0:01:53.814 160M / 160M INFO K-mer Counting (kmer_data.cpp : 308) Estimated 241188234 distinct kmers
0:01:53.828 32M / 160M INFO K-mer Counting (kmer_data.cpp : 313) Filtering singleton k-mers
0:01:54.150 724M / 724M INFO K-mer Counting (kmer_data.cpp : 319) Processing /Users/mars/Desktop/spades_single_end/fastq/R1.fastq
0:04:27.200 724M / 724M INFO K-mer Counting (kmer_data.cpp : 328) Processed 17262970 reads
0:04:27.200 724M / 724M INFO K-mer Counting (kmer_data.cpp : 319) Processing /Users/mars/Desktop/spades_single_end/fastq/R2.fastq
0:07:02.500 724M / 724M INFO K-mer Counting (kmer_data.cpp : 328) Processed 34525940 reads
0:07:02.500 724M / 724M INFO K-mer Counting (kmer_data.cpp : 333) Total 34525940 reads processed
0:07:02.500 724M / 724M INFO K-mer Index Building (kmer_index_builder.hpp : 428) Building kmer index
0:07:02.500 724M / 724M INFO K-mer Splitting (kmer_data.cpp : 91) Splitting kmer instances into 128 buckets. This might take a while.
0:07:02.500 724M / 724M INFO General (file_limit.hpp : 30) Open file limit set to 256
0:07:02.500 724M / 724M INFO General (kmer_index_builder.hpp : 108) Memory available for splitting buffers: 10.3872 Gb
0:07:02.500 724M / 724M INFO General (kmer_index_builder.hpp : 116) Using cell size of 524288
0:07:02.501 8G / 8G INFO K-mer Splitting (kmer_data.cpp : 98) Processing /Users/mars/Desktop/spades_single_end/fastq/R1.fastq
0:07:44.102 8G / 8G INFO K-mer Splitting (kmer_data.cpp : 108) Processed 3636514 reads
0:08:25.846 8G / 8G INFO K-mer Splitting (kmer_data.cpp : 108) Processed 7281705 reads
0:09:08.302 8G / 8G INFO K-mer Splitting (kmer_data.cpp : 108) Processed 10907241 reads
0:09:53.044 8G / 8G INFO K-mer Splitting (kmer_data.cpp : 108) Processed 14540365 reads
0:10:31.197 8G / 8G INFO K-mer Splitting (kmer_data.cpp : 108) Processed 17262970 reads
0:10:31.197 8G / 8G INFO K-mer Splitting (kmer_data.cpp : 98) Processing /Users/mars/Desktop/spades_single_end/fastq/R2.fastq
0:10:54.579 8G / 8G INFO K-mer Splitting (kmer_data.cpp : 108) Processed 19253383 reads
0:11:21.035 8G / 8G INFO K-mer Splitting (kmer_data.cpp : 108) Processed 21251668 reads
0:11:46.696 8G / 8G INFO K-mer Splitting (kmer_data.cpp : 108) Processed 23275865 reads
0:12:12.916 8G / 8G INFO K-mer Splitting (kmer_data.cpp : 108) Processed 25323869 reads
0:12:35.470 8G / 8G INFO K-mer Splitting (kmer_data.cpp : 108) Processed 27337193 reads
0:14:15.936 8G / 8G INFO K-mer Splitting (kmer_data.cpp : 108) Processed 34525940 reads
0:14:15.936 8G / 8G INFO K-mer Splitting (kmer_data.cpp : 113) Total 34525940 reads processed
0:14:16.584 724M / 8G INFO General (kmer_index_builder.hpp : 252) Starting k-mer counting.
0:14:32.137 724M / 8G INFO General (kmer_index_builder.hpp : 258) K-mer counting done. There are 49672548 kmers in total.
0:14:32.138 724M / 8G INFO General (kmer_index_builder.hpp : 260) Merging temporary buckets.
0:14:32.570 724M / 8G INFO K-mer Index Building (kmer_index_builder.hpp : 437) Building perfect hash indices
0:14:37.942 724M / 8G INFO General (kmer_index_builder.hpp : 276) Merging final buckets.
0:14:38.326 724M / 8G INFO K-mer Index Building (kmer_index_builder.hpp : 483) Index built. Total 16229096 bytes occupied (2.61377 bits per kmer).
0:14:38.390 32M / 8G INFO K-mer Counting (kmer_data.cpp : 359) Arranging kmers in hash map order
0:14:43.033 792M / 8G INFO General (main.cpp : 155) Clustering Hamming graph.
0:17:16.522 792M / 8G INFO General (main.cpp : 162) Extracting clusters
0:17:41.626 792M / 8G INFO General (main.cpp : 174) Clustering done. Total clusters: 12605166
0:17:41.668 412M / 8G INFO K-mer Counting (kmer_data.cpp : 381) Collecting K-mer information, this takes a while.
0:17:42.425 1G / 8G INFO K-mer Counting (kmer_data.cpp : 387) Processing /Users/mars/Desktop/spades_single_end/fastq/R1.fastq
0:25:28.606 1G / 8G INFO K-mer Counting (kmer_data.cpp : 387) Processing /Users/mars/Desktop/spades_single_end/fastq/R2.fastq
0:31:56.721 1G / 8G INFO K-mer Counting (kmer_data.cpp : 394) Collection done, postprocessing.
0:31:57.263 1G / 8G INFO K-mer Counting (kmer_data.cpp : 408) There are 49672548 kmers in total. Among them 2096018 (4.21967%) are singletons.
0:31:57.264 1G / 8G INFO General (main.cpp : 180) Subclustering Hamming graph
0:33:17.836 1G / 8G INFO Hamming Subclustering (kmer_cluster.cpp : 649) Subclustering done. Total 226 non-read kmers were generated.
0:33:17.836 1G / 8G INFO Hamming Subclustering (kmer_cluster.cpp : 650) Subclustering statistics:
0:33:17.836 1G / 8G INFO Hamming Subclustering (kmer_cluster.cpp : 651) Total singleton hamming clusters: 7304888. Among them 5557790 (76.0832%) are good
0:33:17.836 1G / 8G INFO Hamming Subclustering (kmer_cluster.cpp : 652) Total singleton subclusters: 344751. Among them 344001 (99.7825%) are good
0:33:17.836 1G / 8G INFO Hamming Subclustering (kmer_cluster.cpp : 653) Total non-singleton subcluster centers: 5838504. Among them 5706857 (97.7452%) are good
0:33:17.836 1G / 8G INFO Hamming Subclustering (kmer_cluster.cpp : 654) Average size of non-trivial subcluster: 7.25687 kmers
0:33:17.836 1G / 8G INFO Hamming Subclustering (kmer_cluster.cpp : 655) Average number of sub-clusters per non-singleton cluster: 1.16659
0:33:17.836 1G / 8G INFO Hamming Subclustering (kmer_cluster.cpp : 656) Total solid k-mers: 11608648
0:33:17.836 1G / 8G INFO Hamming Subclustering (kmer_cluster.cpp : 657) Substitution probabilities: 4,4
0:33:17.847 1G / 8G INFO General (main.cpp : 185) Finished clustering.
0:33:17.847 1G / 8G INFO General (main.cpp : 204) Starting solid k-mers expansion in 8 threads.
0:36:38.764 1G / 8G INFO General (main.cpp : 225) Solid k-mers iteration 0 produced 2627006 new k-mers.
0:39:50.894 1G / 8G INFO General (main.cpp : 225) Solid k-mers iteration 1 produced 551044 new k-mers.
0:42:59.502 1G / 8G INFO General (main.cpp : 225) Solid k-mers iteration 2 produced 11722 new k-mers.
0:46:12.313 1G / 8G INFO General (main.cpp : 225) Solid k-mers iteration 3 produced 877 new k-mers.
0:49:14.087 1G / 8G INFO General (main.cpp : 225) Solid k-mers iteration 4 produced 96 new k-mers.
0:52:16.169 1G / 8G INFO General (main.cpp : 225) Solid k-mers iteration 5 produced 6 new k-mers.
0:52:16.170 1G / 8G INFO General (main.cpp : 229) Solid k-mers finalized
0:52:16.170 1G / 8G INFO General (hammer_tools.cpp : 211) Starting read correction in 8 threads.
0:52:16.170 1G / 8G INFO General (hammer_tools.cpp : 222) Correcting pair of reads: /Users/mars/Desktop/spades_single_end/fastq/R1.fastq and /Users/mars/Desktop/spades_single_end/fastq/R2.fastq
0:52:18.238 2G / 8G INFO General (hammer_tools.cpp : 166) Prepared batch 0 of 800000 reads.
0:52:31.709 2G / 8G INFO General (hammer_tools.cpp : 175) Processed batch 0
0:52:34.225 2G / 8G INFO General (hammer_tools.cpp : 185) Written batch 0
0:52:35.749 2G / 8G INFO General (hammer_tools.cpp : 166) Prepared batch 1 of 800000 reads.
0:52:49.156 2G / 8G INFO General (hammer_tools.cpp : 175) Processed batch 1
0:52:51.347 2G / 8G INFO General (hammer_tools.cpp : 185) Written batch 1
0:52:52.849 2G / 8G INFO General (hammer_tools.cpp : 166) Prepared batch 2 of 800000 reads.
0:53:06.493 2G / 8G INFO General (hammer_tools.cpp : 175) Processed batch 2
0:53:08.673 2G / 8G INFO General (hammer_tools.cpp : 185) Written batch 2
0:53:10.174 2G / 8G INFO General (hammer_tools.cpp : 166) Prepared batch 3 of 800000 reads.
0:53:23.408 2G / 8G INFO General (hammer_tools.cpp : 175) Processed batch 3
0:53:25.593 2G / 8G INFO General (hammer_tools.cpp : 185) Written batch 3
0:53:27.097 2G / 8G INFO General (hammer_tools.cpp : 166) Prepared batch 4 of 800000 reads.
0:53:40.841 2G / 8G INFO General (hammer_tools.cpp : 175) Processed batch 4
0:53:43.015 2G / 8G INFO General (hammer_tools.cpp : 185) Written batch 4
0:53:44.501 2G / 8G INFO General (hammer_tools.cpp : 166) Prepared batch 5 of 800000 reads.
0:53:58.065 2G / 8G INFO General (hammer_tools.cpp : 175) Processed batch 5
0:54:00.237 2G / 8G INFO General (hammer_tools.cpp : 185) Written batch 5
0:54:01.754 2G / 8G INFO General (hammer_tools.cpp : 166) Prepared batch 6 of 800000 reads.
0:54:15.639 2G / 8G INFO General (hammer_tools.cpp : 175) Processed batch 6
0:54:17.845 2G / 8G INFO General (hammer_tools.cpp : 185) Written batch 6
0:54:19.360 2G / 8G INFO General (hammer_tools.cpp : 166) Prepared batch 7 of 800000 reads.
0:54:32.769 2G / 8G INFO General (hammer_tools.cpp : 175) Processed batch 7
0:54:34.937 2G / 8G INFO General (hammer_tools.cpp : 185) Written batch 7
0:54:36.457 2G / 8G INFO General (hammer_tools.cpp : 166) Prepared batch 8 of 800000 reads.
0:54:50.710 2G / 8G INFO General (hammer_tools.cpp : 175) Processed batch 8
0:54:52.844 2G / 8G INFO General (hammer_tools.cpp : 185) Written batch 8
0:54:54.372 2G / 8G INFO General (hammer_tools.cpp : 166) Prepared batch 9 of 800000 reads.
0:55:08.741 2G / 8G INFO General (hammer_tools.cpp : 175) Processed batch 9
0:55:10.914 2G / 8G INFO General (hammer_tools.cpp : 185) Written batch 9
0:55:12.413 2G / 8G INFO General (hammer_tools.cpp : 166) Prepared batch 10 of 800000 reads.
0:55:26.369 2G / 8G INFO General (hammer_tools.cpp : 175) Processed batch 10
0:55:28.525 2G / 8G INFO General (hammer_tools.cpp : 185) Written batch 10
0:55:30.010 2G / 8G INFO General (hammer_tools.cpp : 166) Prepared batch 11 of 800000 reads.
0:55:43.282 2G / 8G INFO General (hammer_tools.cpp : 175) Processed batch 11
0:55:45.414 2G / 8G INFO General (hammer_tools.cpp : 185) Written batch 11
0:55:46.910 2G / 8G INFO General (hammer_tools.cpp : 166) Prepared batch 12 of 800000 reads.
0:56:00.087 2G / 8G INFO General (hammer_tools.cpp : 175) Processed batch 12
0:56:02.279 2G / 8G INFO General (hammer_tools.cpp : 185) Written batch 12
0:56:03.822 2G / 8G INFO General (hammer_tools.cpp : 166) Prepared batch 13 of 800000 reads.
0:56:17.640 2G / 8G INFO General (hammer_tools.cpp : 175) Processed batch 13
0:56:19.807 2G / 8G INFO General (hammer_tools.cpp : 185) Written batch 13
0:56:21.321 2G / 8G INFO General (hammer_tools.cpp : 166) Prepared batch 14 of 800000 reads.
0:56:35.051 2G / 8G INFO General (hammer_tools.cpp : 175) Processed batch 14
0:56:37.195 2G / 8G INFO General (hammer_tools.cpp : 185) Written batch 14
0:56:38.709 2G / 8G INFO General (hammer_tools.cpp : 166) Prepared batch 15 of 800000 reads.
0:56:51.090 2G / 8G INFO General (hammer_tools.cpp : 175) Processed batch 15
0:56:53.193 2G / 8G INFO General (hammer_tools.cpp : 185) Written batch 15
0:56:54.718 2G / 8G INFO General (hammer_tools.cpp : 166) Prepared batch 16 of 800000 reads.
0:57:07.045 2G / 8G INFO General (hammer_tools.cpp : 175) Processed batch 16
0:57:09.204 2G / 8G INFO General (hammer_tools.cpp : 185) Written batch 16
0:57:10.688 2G / 8G INFO General (hammer_tools.cpp : 166) Prepared batch 17 of 800000 reads.
0:57:22.746 2G / 8G INFO General (hammer_tools.cpp : 175) Processed batch 17
0:57:24.886 2G / 8G INFO General (hammer_tools.cpp : 185) Written batch 17
0:57:26.397 2G / 8G INFO General (hammer_tools.cpp : 166) Prepared batch 18 of 800000 reads.
0:57:39.180 2G / 8G INFO General (hammer_tools.cpp : 175) Processed batch 18
0:57:41.323 2G / 8G INFO General (hammer_tools.cpp : 185) Written batch 18
0:57:42.825 2G / 8G INFO General (hammer_tools.cpp : 166) Prepared batch 19 of 800000 reads.
0:57:55.794 2G / 8G INFO General (hammer_tools.cpp : 175) Processed batch 19
0:57:57.923 2G / 8G INFO General (hammer_tools.cpp : 185) Written batch 19
0:57:59.420 2G / 8G INFO General (hammer_tools.cpp : 166) Prepared batch 20 of 800000 reads.
0:58:12.749 2G / 8G INFO General (hammer_tools.cpp : 175) Processed batch 20
0:58:14.970 2G / 8G INFO General (hammer_tools.cpp : 185) Written batch 20
0:58:15.832 2G / 8G INFO General (hammer_tools.cpp : 166) Prepared batch 21 of 462970 reads.
0:58:23.314 2G / 8G INFO General (hammer_tools.cpp : 175) Processed batch 21
0:58:24.554 2G / 8G INFO General (hammer_tools.cpp : 185) Written batch 21
0:58:25.129 1G / 8G INFO General (hammer_tools.cpp : 270) Correction done. Changed 7972253 bases in 4231994 reads.
0:58:25.129 1G / 8G INFO General (hammer_tools.cpp : 271) Failed to correct 53443 bases out of 2933572997.
0:58:25.259 32M / 8G INFO General (main.cpp : 262) Saving corrected dataset description to /Users/mars/Desktop/spades_single_end/fastq/metaspades_output/corrected/corrected.yaml
0:58:25.262 32M / 8G INFO General (main.cpp : 269) All done. Exiting.

== Compressing corrected reads (with gzip)

== Dataset description file was created: /Users/mars/Desktop/spades_single_end/fastq/metaspades_output/corrected/corrected.yaml

===== Read error correction finished.

===== Assembling started.

== Running assembler: K21

0:00:00.000 4M / 4M INFO General (main.cpp : 75) Loading config from /Users/mars/Desktop/spades_single_end/fastq/metaspades_output/K21/configs/config.info
0:00:00.000 4M / 4M INFO General (main.cpp : 75) Loading config from /Users/mars/Desktop/spades_single_end/fastq/metaspades_output/K21/configs/mda_mode.info
0:00:00.000 4M / 4M INFO General (main.cpp : 75) Loading config from /Users/mars/Desktop/spades_single_end/fastq/metaspades_output/K21/configs/meta_mode.info
0:00:00.000 4M / 4M INFO General (memory_limit.hpp : 47) Memory limit set to 250 Gb
0:00:00.000 4M / 4M INFO General (main.cpp : 88) Starting SPAdes, built from N/A, git revision N/A
0:00:00.000 4M / 4M INFO General (main.cpp : 89) Maximum k-mer length: 128
0:00:00.000 4M / 4M INFO General (main.cpp : 90) Assembling dataset (/Users/mars/Desktop/spades_single_end/fastq/metaspades_output/dataset.info) with K=21
0:00:00.000 4M / 4M INFO General (launch.hpp : 51) SPAdes started
0:00:00.000 4M / 4M INFO General (launch.hpp : 58) Starting from stage: construction
0:00:00.000 4M / 4M INFO General (launch.hpp : 61) Two-step RR enabled: 0
0:00:00.000 4M / 4M INFO StageManager (stage.cpp : 126) STAGE == Construction
0:00:00.000 4M / 4M INFO General (read_converter.hpp : 84) Converting reads to binary format for library #0 (takes a while)
0:00:00.000 4M / 4M INFO General (read_converter.hpp : 85) Converting paired reads
0:00:00.170 76M / 76M INFO General (binary_converter.hpp : 139) 16384 reads processed
0:00:00.285 84M / 84M INFO General (binary_converter.hpp : 139) 32768 reads processed
0:00:00.522 100M / 100M INFO General (binary_converter.hpp : 139) 65536 reads processed
0:00:01.012 132M / 132M INFO General (binary_converter.hpp : 139) 131072 reads processed
0:00:01.975 200M / 200M INFO General (binary_converter.hpp : 139) 262144 reads processed
0:00:04.129 240M / 240M INFO General (binary_converter.hpp : 139) 524288 reads processed
0:00:08.432 268M / 268M INFO General (binary_converter.hpp : 139) 1048576 reads processed
0:00:16.662 272M / 276M INFO General (binary_converter.hpp : 139) 2097152 reads processed
0:00:33.577 276M / 276M INFO General (binary_converter.hpp : 139) 4194304 reads processed
0:01:07.809 272M / 280M INFO General (binary_converter.hpp : 139) 8388608 reads processed
0:02:15.197 268M / 280M INFO General (binary_converter.hpp : 139) 16777216 reads processed
0:02:17.302 236M / 280M INFO General (binary_converter.hpp : 159) 17037115 reads written
0:02:17.536 4M / 280M INFO General (read_converter.hpp : 94) Converting single reads
0:02:17.687 136M / 280M INFO General (binary_converter.hpp : 139) 16384 reads processed
0:02:17.740 140M / 280M INFO General (binary_converter.hpp : 139) 32768 reads processed
0:02:17.846 148M / 280M INFO General (binary_converter.hpp : 139) 65536 reads processed
0:02:18.055 164M / 280M INFO General (binary_converter.hpp : 139) 131072 reads processed
0:02:18.428 184M / 280M INFO General (binary_converter.hpp : 159) 205439 reads written
0:02:18.556 4M / 280M INFO General (graph_construction.hpp : 120) Constructing DeBruijn graph for k=21
0:02:18.556 4M / 280M INFO General (kmer_splitters.hpp : 129) Splitting kmer instances into 64 buckets. This might take a while.
0:02:18.556 4M / 280M INFO General (file_limit.hpp : 30) Open file limit set to 256
0:02:18.556 4M / 280M INFO General (kmer_index_builder.hpp : 108) Memory available for splitting buffers: 10.4165 Gb
0:02:18.556 4M / 280M INFO General (kmer_index_builder.hpp : 116) Using cell size of 1048576
0:02:39.440 6G / 6G INFO General (kmer_splitters.hpp : 153) Processed 11804580 reads
0:03:00.695 6G / 6G INFO General (kmer_splitters.hpp : 153) Processed 23708251 reads
0:03:22.223 6G / 6G INFO General (kmer_splitters.hpp : 153) Processed 35691964 reads
0:03:43.328 6G / 6G INFO General (kmer_splitters.hpp : 153) Processed 47533456 reads
0:04:05.050 6G / 6G INFO General (kmer_splitters.hpp : 153) Processed 59439178 reads
0:04:21.389 6G / 6G INFO General (kmer_splitters.hpp : 153) Processed 68559338 reads
0:04:21.389 6G / 6G INFO General (kmer_splitters.hpp : 159) Adding contigs from previous K
0:04:21.626 32M / 6G INFO General (kmer_splitters.hpp : 172) Used 68559338 reads. Maximum read length 85
0:04:21.626 32M / 6G INFO General (kmer_splitters.hpp : 173) Average read length 84.9954
0:04:21.626 32M / 6G INFO General (kmer_index_builder.hpp : 252) Starting k-mer counting.
0:04:26.046 32M / 6G INFO General (kmer_index_builder.hpp : 258) K-mer counting done. There are 45935316 kmers in total.
0:04:26.046 32M / 6G INFO General (kmer_index_builder.hpp : 260) Merging temporary buckets.
0:04:26.428 32M / 6G INFO K-mer Index Building (kmer_index_builder.hpp : 428) Building kmer index
0:04:26.428 32M / 6G INFO General (kmer_splitters.hpp : 285) Splitting kmer instances into 128 buckets. This might take a while.
0:04:26.428 32M / 6G INFO General (file_limit.hpp : 30) Open file limit set to 256
0:04:26.428 32M / 6G INFO General (kmer_index_builder.hpp : 108) Memory available for splitting buffers: 10.4154 Gb
0:04:26.428 32M / 6G INFO General (kmer_index_builder.hpp : 116) Using cell size of 524288
0:04:41.306 8G / 8G INFO General (kmer_splitters.hpp : 304) Processed 45935316 kmers
0:04:41.306 8G / 8G INFO General (kmer_splitters.hpp : 309) Used 45935316 kmers.
0:04:41.373 32M / 8G INFO General (kmer_index_builder.hpp : 252) Starting k-mer counting.
0:04:43.859 32M / 8G INFO General (kmer_index_builder.hpp : 258) K-mer counting done. There are 44558213 kmers in total.
0:04:43.859 32M / 8G INFO General (kmer_index_builder.hpp : 260) Merging temporary buckets.
0:04:44.232 32M / 8G INFO K-mer Index Building (kmer_index_builder.hpp : 437) Building perfect hash indices
0:04:47.465 32M / 8G INFO General (kmer_index_builder.hpp : 276) Merging final buckets.
0:04:47.788 32M / 8G INFO K-mer Index Building (kmer_index_builder.hpp : 483) Index built. Total 14558136 bytes occupied (2.61377 bits per kmer).
0:04:47.816 76M / 8G INFO DeBruijnExtensionIndexBu (kmer_extension_index_build: 85) Building k-mer extensions from k+1-mers
0:04:56.526 76M / 8G INFO DeBruijnExtensionIndexBu (kmer_extension_index_build: 89) Building k-mer extensions from k+1-mers finished.
0:04:56.527 76M / 8G INFO Early tip clipping (early_simplification.hpp : 181) Early tip clipping
0:05:16.064 76M / 8G INFO Early tip clipping (early_simplification.hpp : 184) 14791162 22-mers were removed by early tip clipper
0:05:16.064 76M / 8G INFO General (graph_construction.hpp : 136) Condensing graph
0:05:16.064 76M / 8G INFO UnbranchingPathExtractor (debruijn_graph_constructor: 340) Extracting unbranching paths
0:05:47.284 864M / 8G INFO UnbranchingPathExtractor (debruijn_graph_constructor: 354) Extracting unbranching paths finished. 7287052 sequences extracted
0:05:51.743 864M / 8G INFO UnbranchingPathExtractor (debruijn_graph_constructor: 313) Collecting perfect loops
0:06:02.601 864M / 8G INFO UnbranchingPathExtractor (debruijn_graph_constructor: 329) Collecting perfect loops finished. 72 loops collected
0:06:06.431 1G / 8G INFO General (graph_construction.hpp : 141) Building index with from graph
0:06:06.431 1G / 8G INFO K-mer Index Building (kmer_index_builder.hpp : 428) Building kmer index
0:06:06.431 1G / 8G INFO General (kmer_splitters.hpp : 214) Splitting kmer instances into 16 buckets. This might take a while.
0:06:06.431 1G / 8G INFO General (file_limit.hpp : 30) Open file limit set to 256
0:06:06.431 1G / 8G INFO General (kmer_index_builder.hpp : 108) Memory available for splitting buffers: 82.8607 Gb
0:06:06.431 1G / 8G INFO General (kmer_index_builder.hpp : 116) Using cell size of 4194304
0:06:13.796 1G / 8G INFO General (kmer_splitters.hpp : 225) Processed 7287124 edges
0:06:13.796 1G / 8G INFO General (kmer_splitters.hpp : 230) Used 7287124 sequences.
0:06:13.816 1G / 8G INFO General (kmer_index_builder.hpp : 252) Starting k-mer counting.
0:06:19.147 1G / 8G INFO General (kmer_index_builder.hpp : 258) K-mer counting done. There are 31144154 kmers in total.
0:06:19.147 1G / 8G INFO General (kmer_index_builder.hpp : 260) Merging temporary buckets.
0:06:19.401 1G / 8G INFO K-mer Index Building (kmer_index_builder.hpp : 437) Building perfect hash indices
0:06:28.047 1G / 8G INFO General (kmer_index_builder.hpp : 276) Merging final buckets.
0:06:28.247 1G / 8G INFO K-mer Index Building (kmer_index_builder.hpp : 483) Index built. Total 10175528 bytes occupied (2.61379 bits per kmer).
0:06:28.827 2G / 8G INFO General (edge_index_builders.hpp : 27) Collecting k-mer coverage information from graph, this takes a while.
0:06:31.463 2G / 8G INFO General (edge_index.hpp : 91) Index refilled
0:06:31.535 2G / 8G INFO General (graph_construction.hpp : 173) Filling coverage index
0:06:31.535 2G / 8G INFO General (edge_index_builders.hpp : 105) Collecting k-mer coverage information from reads, this takes a while.
0:11:02.962 2G / 8G INFO General (graph_construction.hpp : 175) Filling coverage and flanking coverage from index
0:11:07.552 2G / 8G INFO General (construction.cpp : 30) Figured out: read length = 85
0:11:07.552 2G / 8G INFO StageManager (stage.cpp : 126) STAGE == EC Threshold Finding
0:11:15.903 2G / 8G INFO ThresholdFinder (ec_threshold_finder.hpp : 114) Bucket size: 25
0:11:15.904 2G / 8G INFO General (genomic_info_filler.cpp : 104) Average edge coverage: 69.9561
0:11:15.904 2G / 8G INFO General (genomic_info_filler.cpp : 105) Graph threshold: 115
0:11:21.318 2G / 8G INFO General (genomic_info_filler.cpp : 147) EC coverage threshold value was calculated as 69.9561
0:11:21.318 2G / 8G INFO General (genomic_info_filler.cpp : 148) Trusted kmer low bound: 0
0:11:21.318 2G / 8G INFO StageManager (stage.cpp : 126) STAGE == Simplification
0:11:21.381 1G / 8G INFO General (simplification.cpp : 380) Graph simplification started
0:11:21.382 1G / 8G INFO General (simplification.cpp : 76) PROCEDURE == InitialCleaning
0:11:21.382 1G / 8G INFO General (graph_simplification.hpp : 645) Flanking coverage based disconnection disabled
0:11:21.382 1G / 8G INFO General (simplification.cpp : 349) Running Self conjugate edge remover
0:11:21.670 1G / 8G INFO General (simplification.cpp : 351) Triggered 0 times
0:11:21.670 1G / 8G INFO General (simplification.cpp : 349) Running Initial isolated edge remover
0:11:22.185 1G / 8G INFO General (simplification.cpp : 351) Triggered 17442 times
0:11:22.185 1G / 8G INFO General (simplification.cpp : 349) Running Initial tip clipper
0:11:23.784 1G / 8G INFO General (simplification.cpp : 351) Triggered 62685 times
0:11:23.784 1G / 8G INFO General (simplification.cpp : 349) Running Initial ec remover
0:11:24.303 1G / 8G INFO General (simplification.cpp : 351) Triggered 0 times
0:11:24.331 1G / 8G INFO General (graph_simplification.hpp : 617) Creating parallel br instance
0:11:24.331 1G / 8G INFO General (simplification.cpp : 403) PROCEDURE == Simplification cycle, iteration 1
0:11:24.331 1G / 8G INFO General (simplification.cpp : 349) Running Tip clipper
0:11:24.884 1G / 8G INFO General (simplification.cpp : 351) Triggered 2053 times
0:11:24.884 1G / 8G INFO General (simplification.cpp : 349) Running Bulge remover
0:15:46.949 1G / 8G INFO General (simplification.cpp : 351) Triggered 63838 times
0:15:46.950 1G / 8G INFO General (simplification.cpp : 349) Running Low coverage edge remover
0:15:49.233 1G / 8G INFO General (simplification.cpp : 351) Triggered 0 times
0:15:49.233 1G / 8G INFO General (simplification.cpp : 403) PROCEDURE == Simplification cycle, iteration 2
0:15:49.233 1G / 8G INFO General (simplification.cpp : 349) Running Tip clipper
0:15:49.645 1G / 8G INFO General (simplification.cpp : 351) Triggered 1069 times
0:15:49.645 1G / 8G INFO General (simplification.cpp : 349) Running Bulge remover
0:15:49.723 1G / 8G INFO General (simplification.cpp : 351) Triggered 29 times
0:15:49.723 1G / 8G INFO General (simplification.cpp : 349) Running Low coverage edge remover
0:16:47.286 1G / 8G INFO General (simplification.cpp : 351) Triggered 1062524 times
0:16:47.286 1G / 8G INFO General (simplification.cpp : 403) PROCEDURE == Simplification cycle, iteration 3
0:16:47.286 1G / 8G INFO General (simplification.cpp : 349) Running Tip clipper
0:16:48.026 1G / 8G INFO General (simplification.cpp : 351) Triggered 9374 times
0:16:48.026 1G / 8G INFO General (simplification.cpp : 349) Running Bulge remover
0:17:27.471 1G / 8G INFO General (simplification.cpp : 351) Triggered 54962 times
0:17:27.471 1G / 8G INFO General (simplification.cpp : 349) Running Low coverage edge remover
0:17:32.639 1G / 8G INFO General (simplification.cpp : 351) Triggered 83619 times
0:17:32.639 1G / 8G INFO General (simplification.cpp : 403) PROCEDURE == Simplification cycle, iteration 4
0:17:32.639 1G / 8G INFO General (simplification.cpp : 349) Running Tip clipper
0:17:32.787 1G / 8G INFO General (simplification.cpp : 351) Triggered 382 times
0:17:32.787 1G / 8G INFO General (simplification.cpp : 349) Running Bulge remover
0:17:44.333 1G / 8G INFO General (simplification.cpp : 351) Triggered 7856 times
0:17:44.333 1G / 8G INFO General (simplification.cpp : 349) Running Low coverage edge remover
0:17:44.334 1G / 8G INFO General (simplification.cpp : 351) Triggered 11 times
0:17:44.334 1G / 8G INFO General (simplification.cpp : 403) PROCEDURE == Simplification cycle, iteration 5
0:17:44.334 1G / 8G INFO General (simplification.cpp : 349) Running Tip clipper
0:17:44.358 1G / 8G INFO General (simplification.cpp : 351) Triggered 6 times
0:17:44.358 1G / 8G INFO General (simplification.cpp : 349) Running Bulge remover
0:17:44.365 1G / 8G INFO General (simplification.cpp : 351) Triggered 2 times
0:17:44.365 1G / 8G INFO General (simplification.cpp : 349) Running Low coverage edge remover
0:17:44.365 1G / 8G INFO General (simplification.cpp : 351) Triggered 0 times
0:17:44.365 1G / 8G INFO General (simplification.cpp : 403) PROCEDURE == Simplification cycle, iteration 6
0:17:44.365 1G / 8G INFO General (simplification.cpp : 349) Running Tip clipper
0:17:44.365 1G / 8G INFO General (simplification.cpp : 351) Triggered 0 times
0:17:44.365 1G / 8G INFO General (simplification.cpp : 349) Running Bulge remover
0:17:44.365 1G / 8G INFO General (simplification.cpp : 351) Triggered 0 times
0:17:44.365 1G / 8G INFO General (simplification.cpp : 349) Running Low coverage edge remover
0:17:44.365 1G / 8G INFO General (simplification.cpp : 351) Triggered 0 times
0:17:44.365 1G / 8G INFO General (simplification.cpp : 162) PROCEDURE == Post simplification
0:17:44.365 1G / 8G INFO General (graph_simplification.hpp : 617) Creating parallel br instance
0:17:44.365 1G / 8G INFO General (simplification.cpp : 294) Iteration 0
0:17:44.365 1G / 8G INFO General (simplification.cpp : 349) Running Relative coverage component remover
0:17:55.094 1016M / 8G INFO General (simplification.cpp : 351) Triggered 74153 times
0:17:55.094 1016M / 8G INFO General (simplification.cpp : 349) Running Disconnecting edges with relatively low coverage

== Error == system call for: "['/usr/local/Cellar/spades/3.10.1/bin/spades', '/Users/mars/Desktop/spades_single_end/fastq/metaspades_output/K21/configs/config.info', '/Users/mars/Desktop/spades_single_end/fastq/metaspades_output/K21/configs/mda_mode.info', '/Users/mars/Desktop/spades_single_end/fastq/metaspades_output/K21/configs/meta_mode.info']" finished abnormally, err code: -10

In case you have troubles running SPAdes, you can write to [email protected]
Please provide us with params.txt and spades.log files from the output directory.

Thanks!

Hi there,

It's a bit confusing to have essentially indential files in the bin/ directory and both have executable permissions on them. The main problem for us is that we use lmod enviromental modules system on our servers and when I module load spades I get both spades and spades.py in my enviroments and both are executable. Naturally I run spades which errors out

Exception caught basic_string::_S_construct null not valid

spades.py works fine obviously

Would you be able to fix this some how?

The best solution in my view is to rename either of files into something else e.g spades -> spades_assembly Ideally it'll be great greatt to have spades.py -> spades so that when I module load spades I get spades. I can move sapdes.py -> spades on our system that's not a problem, but if I do that right now I get a conflict.

Please let me know if you have any temproral fixe for this problem and your long term thought

Thanks

While building Debian packages for SPAdes, we noticed that an extra line is necessary in src/common/utils/segfault_handler.hpp to make building with GCC 7's g++ possible:

--- a/src/common/utils/segfault_handler.hpp
+++ b/src/common/utils/segfault_handler.hpp
@@ -12,6 +12,7 @@
 #include "boost/noncopyable.hpp"
 
 #include <signal.h>
+#include <functional>
 
 struct segfault_handler : boost::noncopyable {
     typedef std::function<void()> callback_t;

Please also see https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=853666 for the original Debian bug report.

Hi,
I'm trying to run SPAdes, but it stops at the read error correction step with err code -9. You can find SPAdes log here.
I used ENA's fastq files from ERR026213 run accession with the following command: spades.py --careful --pe1-1 ERR026213_1.fastq.gz --pe1-2 ERR026213_2.fastq.gz --threads 1 --memory 20 -o ./spades
Is there anything that I can do to fix this?
Thanks in advance.
Best regards,
Miguel Machado

I have a problem with metaspades. I am running a metagenome assemble and Im getting this error.

finished abnormally, err code: 255 in python

And this is my command:

metaspades.py -1 PM2_forward.Tp.fastq -2 PM2_reverse.Tp.fastq -k 21,31,41,51,61 -o spades_PM2_out

Any ideas?

Hello,

Is SPAdes 3.11.1 MPI, or some other form of distributed computing, compliant? I'm running out of memory when I try to run my assembly on a single node in my cluster and can't find instructions for how to run it with the entire cluster's resources instead of a single node's.

• Seth

Hi,

Are there any method to increase the maximum k-mer size beyond the default (127)?

Sometimes, I perform sequencing with 2*300 bp read length. And, I'd like to test the effect of increasing maximum k-mer size beyond 127 (e.g., 151) on assembly quality.

Thanks.

Hello,
I'm trying to assemble plasmid sequences converted from sanger sequencing reads using the --plasmid mode but get an error. Here is a notebook showing the steps I took and here is a gist with the params and log. Do you know what I should change about my run parameters?

I was able to successfully run velvet on the same sequences.
Warmest,
Olga

I'm seeing this error message when compiling SPAdes 3.11.0 on macOS with gcc 7.2.0 (installed using Homebrew).

make -f projects/ionhammer/CMakeFiles/kmer_evaluator.dir/build.make projects/ionhammer/CMakeFiles/kmer_evaluator.dir/build
[ 80%] Building CXX object projects/ionhammer/CMakeFiles/kmer_evaluator.dir/kmer_data.cpp.o
cd /tmp/spades-20170906-42729-c5stra/SPAdes-3.11.0/src/build/projects/ionhammer && /Users/sjackman/.homebrew/Library/Homebrew/shims/super/g++-7  -DNDEBUG -DUSE_GLIBCXX_PARALLEL=1 -I/tmp/spades-20170906-42729-c5stra/SPAdes-3.11.0/src/build/projects/ionhammer -I/tmp/spades-20170906-42729-c5stra/SPAdes-3.11.0/src/projects/ionhammer -I/tmp/spades-20170906-42729-c5stra/SPAdes-3.11.0/src/include -I/tmp/spades-20170906-42729-c5stra/SPAdes-3.11.0/src/build/include -I/tmp/spades-20170906-42729-c5stra/SPAdes-3.11.0/src -I/tmp/spades-20170906-42729-c5stra/SPAdes-3.11.0/src/common -isystem /tmp/spades-20170906-42729-c5stra/SPAdes-3.11.0/src/../ext/include -isystem /Users/sjackman/.homebrew/include  -fopenmp -std=c++11 -DNDEBUG   -Wno-deprecated -g0 -O2 -Wall -Wextra -Wconversion -Wno-sign-conversion -Wno-long-long -Wwrite-strings -o CMakeFiles/kmer_evaluator.dir/kmer_data.cpp.o -c /tmp/spades-20170906-42729-c5stra/SPAdes-3.11.0/src/projects/ionhammer/kmer_data.cpp
In file included from /tmp/spades-20170906-42729-c5stra/SPAdes-3.11.0/src/projects/ionhammer/kmer_data.hpp:12:0,
                 from /tmp/spades-20170906-42729-c5stra/SPAdes-3.11.0/src/projects/ionhammer/kmer_data.cpp:8:
/tmp/spades-20170906-42729-c5stra/SPAdes-3.11.0/src/projects/ionhammer/config_struct.hpp:46:3: error: 'uint' does not name a type; did you mean 'int'?
   uint max_full_del = 1;
   ^~~~

Here's a gist of the logs:
https://gist.github.com/sjackman/e6f3545a556d23a8c3cc80b359331141#file-02-make-L694

I've noticed that a spades assembly I've been running has been spending a long period of time in the path_extender.hpp stage, and that it's only using a single thread for this. Would there be a way to parallelise this?

The command I am using is:

/usr/local/bin/spades.py 
  --pe1-12 /tmp/tmp.fyFUUmBILZ/unmerged.fq.gz
  --s2 /tmp/tmp.AzsDG8zqLS/merged.fq.gz
  -o /tmp/tmp.rzr9ZjS78c
  --threads 8
  --phred-offset  33 
  --cov-cutoff  auto
  --careful
  -k 25,55,95

Dear Developer,
I am using SPADES 3.10 for 50X illumina and 10X pacbio assembly. The genome size is 6G and my RAM is 1T.
My command line is:
spades.py --careful -t 24 -m 980 --dataset dataset.yaml -o SPADES_out

However, I got error below:
7:16:41.441 13G / 13G INFO K-mer Splitting (kmer_data.cpp : 113) Total 1426550508 reads processed
7:16:42.959 96M / 13G INFO General (kmer_index_builder.hpp : 252) Starting k-mer counting.
8:40:59.337 96M / 13G INFO General (kmer_index_builder.hpp : 258) K-mer counting done. There are 24720520342 kmers in total.
8:40:59.337 96M / 13G INFO General (kmer_index_builder.hpp : 260) Merging temporary buckets.
8:48:55.827 96M / 13G INFO K-mer Index Building (kmer_index_builder.hpp : 437) Building perfect hash indices
8:48:55.828 96M / 13G WARN K-mer Index Building (kmer_index_builder.hpp : 451) Number of threads was limited down to 12 in order to fit the memory limits during the index
9:28:28.780 7G / 462G INFO General (kmer_index_builder.hpp : 276) Merging final buckets.
9:34:04.745 7G / 462G INFO K-mer Index Building (kmer_index_builder.hpp : 483) Index built. Total 8076657616 bytes occupied (2.61375 bits per kmer).
9:34:04.747 7G / 462G ERROR K-mer Counting (kmer_data.cpp : 356) The reads contain too many k-mers to fit into available memory. You need approx. 920.911GB o

== Error == system call for: "['/home/zhangxt/software/SPAdes-3.10.0-Linux/bin/hammer', '/share/bioinfo/zhangxt/project/12_SES208_SPADES/SPADES_out/corrected/configs/config.info']" finishe

======= SPAdes pipeline finished abnormally and WITH WARNINGS!

=== Error correction and assembling warnings:

• 8:48:55.828 96M / 13G WARN K-mer Index Building (kmer_index_builder.hpp : 451) Number of threads was limited down to 12 in order to fit the memory limits during the index
  ======= Warnings saved to /share/bioinfo/zhangxt/project/12_SES208_SPADES/SPADES_out/warnings.log

=== ERRORs:

• 9:34:04.747 7G / 462G ERROR K-mer Counting (kmer_data.cpp : 356) The reads contain too many k-mers to fit into available memory. You need approx. 920.911GB
• system call for: "['/home/zhangxt/software/SPAdes-3.10.0-Linux/bin/hammer', '/share/bioinfo/zhangxt/project/12_SES208_SPADES/SPADES_out/corrected/configs/config.info']" finished abnormal

I though my memory should be enough. Is there any way to solve this problem? Thanks so much!

Hi, I have been having some errors when I tried to submit the job to a slurm cluster. It seems to run fine if I start the assembly on login lode. However, when I submit the job to standard node, it fails with the follows error:
"== Error == file not found......(right reads, library number: 1, library type: paired-end)"

There were not params.txt or spades.log files created. This error is printed to the stdout.

here's the end of the log file before it crashes, any help would be much appreciated!

  0:41:02.267     5G / 7G    INFO    General                 (kmer_splitters.hpp        : 225)   Processed 159541 edges
  0:41:06.135     5G / 7G    INFO    General                 (kmer_splitters.hpp        : 225)   Processed 320173 edges
  0:41:09.491     5G / 7G    INFO    General                 (kmer_splitters.hpp        : 225)   Processed 457944 edges
  0:41:09.491     5G / 7G    INFO    General                 (kmer_splitters.hpp        : 230)   Used 457944 sequences.
  0:41:09.537     4G / 7G    INFO    General                 (kmer_index_builder.hpp    : 252)   Starting k-mer counting.
  0:41:23.481     4G / 7G    INFO    General                 (kmer_index_builder.hpp    : 258)   K-mer counting done. There are 52636108 kmers in total.
  0:41:23.481     4G / 7G    INFO    General                 (kmer_index_builder.hpp    : 260)   Merging temporary buckets.
  0:41:24.686     4G / 7G    INFO   K-mer Index Building     (kmer_index_builder.hpp    : 437)   Building perfect hash indices
  0:41:41.979     4G / 7G    INFO    General                 (kmer_index_builder.hpp    : 276)   Merging final buckets.
  0:41:43.745     4G / 7G    INFO   K-mer Index Building     (kmer_index_builder.hpp    : 483)   Index built. Total 17197336 bytes occupied (2.61377 bits per kmer).
  0:41:44.566     6G / 7G    INFO    General                 (perfect_hash_map.hpp      : 185)   Arranging kmers in hash map order
  0:42:01.700     6G / 7G    INFO    General                 (perfect_hash_map.hpp      : 197)   Done. Total swaps: 52635845
  0:42:01.701     6G / 7G    INFO    General                 (edge_index_builders.hpp   :  27)   Collecting k-mer coverage information from graph, this takes a while.
  0:42:15.050    11G / 11G   INFO   PacIndex                 (pac_index.hpp             :  85)   Index constructed
  0:42:19.995    12G / 12G   INFO    General                 (hybrid_aligning.cpp       : 315)   Prepared batch 0 of 50000 reads.

== Error ==  system call for: "['/Volumes/work/cryomics/apps/SPAdes-3.10.1-Darwin/bin/spades', '/Volumes/work/cryomics/seq_data/Sphaeroforma_sirkka_B5/minION_B5_20170505_great/spades_hybrid_all/K55/configs/config.info', '/Volumes/work/cryomics/seq_data/Sphaeroforma_sirkka_B5/minION_B5_20170505_great/spades_hybrid_all/K55/configs/careful_mode.info']" finished abnormally, err code: -10

params.txt
spades.log.txt

While trying to run a mammalian (chromosome-specific) assembly with ~20 million reads, stages with k=21 and k=33 complete successfully. Subsequently, the k=55 step fails with

<jemalloc>: Error in malloc(): out of memory. Requested: 64, active: 210579226624

I've tried playing with the memory parameters, setting -m to 120, 256, 500 etc. but this continues to throw an error without completion.

Could you provide any ideas on how to determine optimal -m ? The node I'm using has 64 processors and max of 512 GB memory.

Would reducing the number of threads help ? How does SPAdes decide the total RAM requested ?
Can I try any other parameters to get this assembly to complete? I'm open to any suggestions.

Parameters :

System information: SPAdes version: 3.6.1 Threads: 64 Memory limit (in Gb): 250 Python version: 2.7.13 OS: Linux-3.2.0-4-amd64-x86_64-with-debian-7.5 k: [21, 33, 55] Mismatch careful mode is turned OFF Repeat resolution is enabled MismatchCorrector will be SKIPPED Coverage cutoff is turned OFF

Is there a way to execute spades by reading form a data stream?

Eg:

zcat reads.fq.gz | spades.py --12 -

or

zcat reads.fq.gz | spades.py --12 /dev/stdin

If not, any intentions of implementing it?

Hi,

I have run spades 3.11 without error correction with wide range of k-mers (19, 21, 23, 25, 27, 31, 35, 39, 41, 43, 45, 47, 51, 55, 59, 61, 63, 67, 71, 81, 91) on 48.8GB large data (47GB interleaved Illumina paired-end reads + 1.8GB Illumina single-end reads).
I was able to successfully get the results for all k-mer values, except for one, 61.
I ran this job twice, and I got the same error:
spades: /spades/src/common/modules/path_extend/weight_counter.hpp:295: virtual double path_extend::CoverageAwareIdealInfoProvider::EstimatePathCoverage(const path_extend::BidirectionalPath&) const: Assertion `path.Length() > 0' failed.
== Error == system call for: "['/util/opt/anaconda/4.3.11/envs/spades-3.11.0/share/spades-3.11.0-0/bin/spades', '/results/spades/311/no_ec/k_61/K61/configs/config.info', '/results/spades/311/no_ec/k_61/K61/configs/mda_mode.info', '/results/spades/311/no_ec/k_61/K61/configs/rna_mode.info']" finished abnormally, err code: -6

The total memory used by the job is ~233GB, and I am requesting 980GB from my cluster with -m 950.
I don't know why this error occurs for this particular k value only, so I would appreciate any help.
Please find the attachments below:
params.txt
spades.log

Thank you,
Natasha

I have multiple fastq files for a sample, i.e. multipe R1 and R2 files. I am wondering what effect it will have on the assembly if I concatenate all R1* and R2* files together and then use them in assembly. To be clear, I here mean

cat R1* > R1_input.fq
cat R2* > R2_input.fq

followed by

spades --pe-1 R1_input.fq --pe-2 R2_input.fq

vs using each R file separately with --pe-1-1. --pe-2-1, --pe-1-2 and --pe-2-2 and so forth.

Hi,

I just downloaded "SPAdes-3.11.0-Linux.tar.gz", and when I try and test the binaries, for "rnaspades.py" and Python 3.2, 3.3, 3.4, 3.5 and 3.6 I get the following error:
$ ./rnaspades.py --test
Traceback (most recent call last):
File "./rnaspades.py", line 1043, in
main(sys.argv)
File "./rnaspades.py", line 609, in main
cfg, dataset_data = fill_cfg(options, log)
File "./rnaspades.py", line 484, in fill_cfg
k_value = support.get_reads_length(dataset_data, log) / 2 - 1
File "/home/centos/install/SPAdes-3.11.0-Linux/share/spades/spades_pipeline/support.py", line 607, in get_reads_length
max_reads_lenghts = [get_max_reads_length(reads_file, log, num_checked) for reads_file in get_reads_files(dataset_data)]
File "/home/centos/install/SPAdes-3.11.0-Linux/share/spades/spades_pipeline/support.py", line 607, in
max_reads_lenghts = [get_max_reads_length(reads_file, log, num_checked) for reads_file in get_reads_files(dataset_data)]
File "/home/centos/install/SPAdes-3.11.0-Linux/share/spades/spades_pipeline/support.py", line 631, in get_max_reads_length
max_reads_length = max([len(rec) for rec in itertools.islice(SeqIO.parse(SeqIO.Open(reads_file, "r"), file_type), num_checked)])
File "/home/centos/install/SPAdes-3.11.0-Linux/share/spades/spades_pipeline/support.py", line 631, in
max_reads_length = max([len(rec) for rec in itertools.islice(SeqIO.parse(SeqIO.Open(reads_file, "r"), file_type), num_checked)])
File "/home/centos/install/SPAdes-3.11.0-Linux/share/spades/spades_pipeline/common/SeqIO.py", line 113, in parse_fastq
assert(rec_id[0] == '@')
AssertionError

The same error occurs when I compile SPAdes 3.11 from the source code.
However, both the binary and the source code work fine with Python 2.6 and Python 2.7.

Can you please help me resolve this issue ?

Thank you,
Natasha

Should --only-error-correction output N-masked sequence?

Now it output the corrected char?

I've never had this fail before, but today it happened.
Even if the FASTQ file has 100,000 reads in it with high qualities eg. 'BDIBDIBDI' it fails.

If I force --phred-offset 33 it works.

% spades.py --version
SPAdes v3.11.0

% cat bug.fq
@genome01_66313_66767_2:0:0_2:0:0_0/1
GACCTGATATTGTGTTATTCATAAATGGATTACCAGTAGTGGTAATTGAACTAAAGAATTCTACAAATGA
+
BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB
@genome01_3674_4142_4:1:0_2:0:0_1/1
AGTCAAAGTGCATCACTTATAGCAAAGTCGTCTACCTCAATATATTGGACACCACAATGTGGTTCTATTT
+
BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB
@genome01_6328_6776_6:0:0_4:0:0_2/1
TACAAGTAAAACTTGAAGATGCGATTGAATCGGACCAAACATTTGAAATGTTAATGGGTGACGTTGTCGC
+
BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB

% spades.py -o /tmp/bug -s #bug.fq

0:00:00.001     4M / 4M    INFO    General                 (memory_limit.hpp          :  51)   Memory limit set to 250 Gb
  0:00:00.001     4M / 4M    INFO    General                 (main.cpp                  :  93)   Trying to determine PHRED offset
  0:00:00.001     4M / 4M    ERROR   General                 (main.cpp                  :  96)   Failed to determine offset! Specify it manually and restart, please!

In your documentation you mention this: Currently metaSPAdes supports only a single library which has to be paired-end (we hope to remove this restriction soon). I was curious what the timeline is for this feature? I currently have a massive amount of single read data and it would be great if I could compare metaspades with the other assemblers I'm testing. Thanks and keep up the great work!

Dear Developer,

I'm getting same errors when I'm trying hybrid assembly.

The errors occur at the same stage (hybrid_aligning.cpp).
At this time, error occurred during processing 100000 reads, but some times it could process more reads (>200000).

I have tried several times after changing data set (with and without mate-pair, PacBio reads).

== Error == system call for: "['//SPAdes-3.11.1-Linux/bin/spades', '//SPADES_data/run6/K61/configs/config.info']" finished abnormally, err code: -11
(I have changed directory name to "***")

I thought it was a memory issue since it stopped when memory usage reached 40-45G.
But assembling only with short Illumina reads seems working well even when it reached 55G (it is still working though).

If you could tell me solutions, I would appreciate it very much.

Thanks,
Sho

params.txt
spades.log

Nothing works and the project is awful!

A while back the --cov-cutoff option appeared, and seemed to be default auto. It is now default off.

We noticed a few strange things happening with auto.

There doesn't seem to be any documentation about it in the manuals.

Can you clarify exactly what it is doing?

Hi,

First, thanks for developing this program. It is straightforward and relatively easy.

I keep running into the same issue though, in which I have assemblies stalling for days to weeks, typically at PathExtender 90% or Gap Closing. I originally tried a few assemblies on a local cluster, and assemblies with ~20gb per forward and reverse read stalled out indefinitely. I have since moved to a larger, more expensive cluster.

I currently am attempting a rather large assembly (forward and reverse reads are each 115 gb) and it has been stuck on PathExtender 90% for over 6 days now. I am running this on a 3 TB node with 42 threads, both of which are specified in my code:

rnaspades.py --only-assembler \
-o spades_full-node \
--threads 42 --memory 3000 -k 35 \
-1 $LOCAL/allreads_1P.fq \
-2 $LOCAL/allreads_2P.fq

Maximum memory used to date on this assembly is ~820 gb, but this has not increased during the final Path Extender step.

Any thoughts on this?

Thanks,
Adam

Hi,
I use spades for hybrid assemble with illumina reads and Pacbio Sequel reads. It reports the warnings as below:

=== Error correction and assembling warnings:
 * 32:42:52.332   107G / 311G  WARN   ScaffoldingUniqueEdgeAna (scaff_supplementary.cpp   :  59)   Less than half of genome in unique edges!
======= Warnings saved to /dat1/Rose_Denovo/resf/spade/ass4/warnings.log

SPAdes log can be found here: /dat1/Rose_Denovo/resf/spade/ass4/spades.log

And the result show the longest scaffolds in the scaffolds.fasta file only 70kb.
Below is the command I used:

/home/data1/spades/bin/spades.py -t 80 --memory 850 -1 ../g1_R1.clean.fq -2 ../g1_R2.clean.fq --pacbio ../../m54061_170321_090037.subreads.bam.fastq -o ass4 >spades.out 2>&1

Dear SPAdes team,
I'm trying to run SPAdes in several libraries using 1 thread and a maximum memory of 1 Gb , but it keeps crashing in K55 with

: Error in malloc(): out of memory. Requested: 392167424, active: 398458880
== Error == system call for: "['/biotools/INNUca/src/SPAdes-3.9.0-Linux/bin/spades', '/home/jani/21_STEC/Test/105400_S9_L001/spades/K55/configs/config.info', '/home/jani/21_STEC/Test/105400_S9_L001/spades/K55/configs/careful_mode.info']" finished abnormally, err code: -6

I don't understand why is SPAdes requiring such an amount of memory even if I set the maximum memory to 1 Gb!
Nevertheless, I'm sure I have around 29 Gb of available memory (20 Gb actually free and 9 Gb cached/buffered).

I attached SPAdes stdout (SPAdes_stdout.txt)

I hope you can help me solving this problem.
Best regards,
Miguel

Hello

while running SPAdes-3.11.0 one of our users reported the following problem.

spades.py --meta -1 file1.fastq -2 file.fastq -o test/ -t 4

ends with the following error

== Running assembler: K21

Exception caught /scratch/eric/test/K21/configs/config.info(3): cannot open include file simplification.info


== Error ==  system call for: "['/mount/gensoft2/exe/SPAdes/3.11.0/bin/spades', '/scratch/eric/test/K21/configs/config.info', '/scratch/eric/test/K21/configs/mda_mode.info', '/scratch/eric/test/K21/configs/meta_mode.info']" finished abnormally, err code: 4

when checking the file config.info from test/configs/debruijn/config.info
one ca notice the following.
incudes are relative.

; input options:

#include "simplification.info"
#include "construction.info"
#include "distance_estimation.info"
#include "detail_info_printer.info"
#include "tsa.info"
#include "pe_params.info"

K               55

after some code reading I ended hacking spades_logic.py (plain old patch attached)
to generate a config.info file that handles the full path for the included files regarding the running configuration directory.

eg it will have something like that.

#include "/mount/tests/SPAdes/3.11.0/test/K21/configs/debruijn/simplification.info"
#include "/mount/tests/SPAdes/3.11.0/test/K21/configs/debruijn/construction.info"
#include "/mount/tests/SPAdes/3.11.0/test/K21/configs/debruijn/distance_estimation.info"
#include "/mount/tests/SPAdes/3.11.0/test/K21/configs/debruijn/detail_info_printer.info"
#include "/mount/tests/SPAdes/3.11.0/test/K21/configs/debruijn/tsa.info"
#include "/mount/tests/SPAdes/3.11.0/test/K21/configs/debruijn/pe_params.info"

instead of just plain file name.

with this patch applied, spades hapilly runs to the normal end and provides expected results

let me know what you think about this patch.

NB as far as I understand the code config files are copied from share/spades/config files.
maybee installer can set those path at install time.

regards

Eric

config_include_patch.txt

I am trying to do a hybrid SPAdes assembly with paired end, mate pair and Nanopore data, but the assembly has the following error:

spades: /spades/src/common/modules/path_extend/path_extender.hpp:353: virtual size_t path_extend::PairedInfoLoopEstimator::EstimateSimpleCycleCount(const path_extend::BidirectionalPath&, path_extend::EdgeId, path_extend::EdgeId) const: Assertion `path.Size() > 1' failed.

== Error == system call for: "['/localdisk/home/s0920593/software/SPAdes-3.11.1-Linux/bin/spades', '/localdisk/data/chlamydomonas/sequencing/c_incerta/hybrid_SPAdes/K55/configs/config.info', '/localdisk/data/chlamydomonas/sequencing/c_incerta/hybrid_SPAdes/K55/configs/careful_mode.info']" finished abnormally, err code: -6

This may be a problem with the mate pair library as I meet the same error with or without the Nanopore data, but not with only the paired end + Nanopore.

Cheers,
Rory

spades.log
params.txt

Hi, I am trying to assemble a plasmid using paired-end Illumina reads from a whole genome sequence with Oxford Nanopore long reads from a plasmid DNA extraction from the same organism.

The command that I used was:

spades.py --careful --plasmid --pe1-1 NR3055_1.fastq.gz --pe1-2 NR3055_2.fastq.gz \
--nanopore NR3055_nano_ltd.fastq -o pNR3055_ltd

SPAdes reads the libraries correctly but then does not seem to incorporate the long reads - I have attached the log. Any advice is much appreciated!

spades_log.docx

Tested on macOS 10.10, 10.11, and 10.12: https://bot.brew.sh/job/Homebrew%20Science%20Pull%20Requests/6988/

The error is

[ 83%] Building CXX object common/utils/CMakeFiles/utils.dir/autocompletion.cpp.o
cd /tmp/spades-20170202-70042-4p7f29/SPAdes-3.10.0/src/build/common/utils && /usr/local/Homebrew/Library/Homebrew/shims/super/g++-6   -DNDEBUG -DUSE_GLIBCXX_PARALLEL=1 -I/tmp/spades-20170202-70042-4p7f29/SPAdes-3.10.0/src/build/common/utils -I/tmp/spades-20170202-70042-4p7f29/SPAdes-3.10.0/src/common/utils -I/tmp/spades-20170202-70042-4p7f29/SPAdes-3.10.0/src/include -I/tmp/spades-20170202-70042-4p7f29/SPAdes-3.10.0/src/build/include -I/tmp/spades-20170202-70042-4p7f29/SPAdes-3.10.0/src -I/tmp/spades-20170202-70042-4p7f29/SPAdes-3.10.0/src/common -isystem /tmp/spades-20170202-70042-4p7f29/SPAdes-3.10.0/src/../ext/include -isystem /usr/local/include  -fopenmp -std=c++11 -DNDEBUG   -Wno-deprecated -g0 -O2 -Wall -Wextra -Wconversion -Wno-sign-conversion -Wno-long-long -Wwrite-strings -o CMakeFiles/utils.dir/autocompletion.cpp.o -c /tmp/spades-20170202-70042-4p7f29/SPAdes-3.10.0/src/common/utils/autocompletion.cpp
/tmp/spades-20170202-70042-4p7f29/SPAdes-3.10.0/src/common/utils/autocompletion.cpp: In function 'char* online_visualization::CommandGenerator(const char*, int)':
/tmp/spades-20170202-70042-4p7f29/SPAdes-3.10.0/src/common/utils/autocompletion.cpp:22:72: error: 'strdup' was not declared in this scope
             char* answer = strdup(list_possible_matches.front().c_str());
                                                                        ^
/tmp/spades-20170202-70042-4p7f29/SPAdes-3.10.0/src/common/utils/autocompletion.cpp:30:45: error: 'strlen' was not declared in this scope
             if (!name.compare(0, strlen(text), text))
                                             ^
make[2]: *** [common/utils/CMakeFiles/utils.dir/autocompletion.cpp.o] Error 1
make[1]: *** [common/utils/CMakeFiles/utils.dir/all] Error 2
make: *** [all] Error 2

As a workaround, I can toss in #include <string.h> just after the #include <string>:

    inreplace "src/common/utils/autocompletion.cpp", /(#include <string>)/,
                                                     "\\1\n#include <string.h>"

I am trying to assemble a viral metagenome using SPAdes. The dataset consists of ~34M paired 2x300 Nextera reads . The original dataset of 60M reads was normalised with bbnorm prior to assembly. SPAdes gets to k99 and then throws this error:

=== Stack Trace ===
[0x40c26a]
[0x68c636]
[0x6807fb]
[0x7a3959]
[0x7ac4fd]
[0x7ee5a9]
spades: /spades/src/common/utils/kmer_mph/kmer_index_builder.hpp:197: size_t utils::KMerDiskCounter<Seq, traits>::MergeKMers(const string&, const string&, unsigned int) [with Seq = RuntimeSeq<128ul>; traits = utils::kmer_index_traits<RuntimeSeq<128ul> >; size_t = long unsigned int; std::string = std::basic_string]: Assertion `std::is_sorted(beg, end, adt::array_less())' failed.

== Error == system call for: "['/gpfs/ts0/home/bt273/tools/SPAdes-3.11.0-Linux/bin/spades', '/gpfs/ts0/home/bt273/long_read_viromes/L4/spades.bbnorm.short.only/K99/configs/config.info', '/gpfs/ts0/home/bt273/long_read_viromes/L4/spades.bbnorm.short.only/K99/configs/mda_mode.info', '/gpfs/ts0/home/bt273/long_read_viromes/L4/spades.bbnorm.short.only/K99/configs/meta_mode.info']" finished abnormally, err code: -6

====================

We have previously successfully assembled this dataset with the previous version of SPAdes, but want to incorporate nanopore reads in a hybrid metagenomic assembly, once we can get the short-read version working. The assembly is running on a HPC node with 3Gb of RAM.

Please advise.

spades.log
params.txt

During a hybrid assembly with Illumina, Pacbio and Nanopore reads, I encountered the following error:

== Error ==  system call for: "['gzip', '-f', '-7', '/gpfs/fs2/scratch/bli16/SPAdes/P212_ai_PB_NP_spades/corrected/P212_ai.read1.00.0_0.cor.fastq']" finished abnormally, err code:
 1

Not sure is it because of some dependcy issues or something else. The related files are attached. Please let me know if you need further info. Thank you.

warnings.log
spades.log
params.txt

Hi,

I am trying to assemble some reads in FASTQ format that were sequenced on a HiSeq 4000 and the quality scores are in Illumina 1.9 format, which have a phred offset of 33. However, when I ran SPAdes, it thought the reads had an offset of 64 and resulted in a very bad assembly. The incorrect auto-detection may be because the ASCII character "K" is a valid score of very high quality for this format of Illumina reads. Fortunately, I caught this and re-ran the assembly with --phred_offset 33 and everything came out great. In a future version it might be helpful to edit the auto-detection to make sure Illumina 1.9 quality scores are auto-detected as phred + 33.

Bryan

There's a bioconda recipe for installing spades on Linux and MacOS:
https://bioconda.github.io/
https://github.com/bioconda/bioconda-recipes/tree/master/recipes/spades
Maybe, mention this installation method in the docs.

Hi there,

I don't believe there's a past issue regarding the error message I get when I run the test dataset. Here are two of the output files.

params.txt
spades.log

Any help with this would be greatly appreciated. Thanks,
Marybel

Thanks for making the spades repo public.

A short README.md explaining the repo would be helpful, perhaps with links to the HTML manuals.

Even better, use gh-pages branch to publish to https://ablab.github.io/spades

Dear developer,
We should use raw PacBio reads or corrected and trimmed PacBio reads when we use Spades for assembling?
Thanks,
Fuyou

The vendored version of jemalloc in SPAdes is outdated and does not include the upstream jemalloc fixes for macOS Sierra (10.12). As a result, SPAdes is currently quite broken on Sierra.

The exception displayed is

malloc: *** malloc_zone_unregister() failed for 0x7fffb1c3c000

and then it hangs indefinitely.

Luckily, this has been fixed upstream in jemalloc/jemalloc#427 and the fix is included in the latest release: https://github.com/jemalloc/jemalloc/releases

So it would be great if you could upgrade the vendored jemalloc so that SPAdes works on macOS 10.12.

A minimal patch to SPAdes is here: https://gist.githubusercontent.com/ilovezfs/931c240e527c55643fd72cb6df87db50/raw/22fcf917a48f434cebdbefe47cd9dd4a78fb64d4/gistfile1.txt

However, I would highly recommend using jemalloc 4.3.1 and not using the minimal patch.

Here is a full log of what the failure looks like:

bash-4.4$ spades.py --test
Command line: /usr/local/bin/spades.py	--test	

System information:
  SPAdes version: 3.9.0
  Python version: 2.7.12
  OS: Darwin-16.1.0-x86_64-i386-64bit

Output dir: /usr/local/Homebrew/Library/Taps/homebrew/homebrew-science/spades_test
Mode: read error correction and assembling
Debug mode is turned OFF

Dataset parameters:
  Multi-cell mode (you should set '--sc' flag if input data was obtained with MDA (single-cell) technology or --meta flag if processing metagenomic dataset)
  Reads:
    Library number: 1, library type: paired-end
      orientation: fr
      left reads: ['/usr/local/Cellar/spades/3.9.0/share/spades/test_dataset/ecoli_1K_1.fq.gz']
      right reads: ['/usr/local/Cellar/spades/3.9.0/share/spades/test_dataset/ecoli_1K_2.fq.gz']
      interlaced reads: not specified
      single reads: not specified
Read error correction parameters:
  Iterations: 1
  PHRED offset will be auto-detected
  Corrected reads will be compressed (with gzip)
Assembly parameters:
  k: automatic selection based on read length
  Repeat resolution is enabled
  Mismatch careful mode is turned OFF
  MismatchCorrector will be SKIPPED
  Coverage cutoff is turned OFF
Other parameters:
  Dir for temp files: /usr/local/Homebrew/Library/Taps/homebrew/homebrew-science/spades_test/tmp
  Threads: 16
  Memory limit (in Gb): 250


======= SPAdes pipeline started. Log can be found here: /usr/local/Homebrew/Library/Taps/homebrew/homebrew-science/spades_test/spades.log


===== Read error correction started. 


== Running read error correction tool: /usr/local/Cellar/spades/3.9.0/bin/hammer /usr/local/Homebrew/Library/Taps/homebrew/homebrew-science/spades_test/corrected/configs/config.info

hammer(13447,0x7fffb1c463c0) malloc: *** malloc_zone_unregister() failed for 0x7fffb1c3c000

Review and comment on SPAdes vs metaSPAdes behavior on the data from http://biorxiv.org/content/early/2017/06/06/120154

I ran spades 3.10 as following:
SPAdes-3.10.0-Linux/bin/spades.py -t 10 -k21,31,51,71 -o spadeswref --trusted-contigs ./virusgt1.fasta -s Hepacivirus.s.fastq --iontorrent

The longest contig I got has this header:
>NODE_1_length_9615_cov_0.109807
And there is no 'N' gap in the contig.
My question is, how can a contig has converage less than 1, equal 0.1? Does it mean spades take sequences from the trusted contigs virusgt1.fasta?

In a similar run, I got longest contig around 3kb while the trusted contig is about 10kb. Therefore I thought spades does not incorporate bases from the trusted contigs. Am I wrong?

Essentially I want to try and use spades as a reference-assisted de novo assembler

Hi all,

I'm using paired-end assembly on GAGE-b MiSeq dataset. (--pe-1 r1.fastq --p2-2 r2.fastq)

In the log file, I found "Skipping processing of scaffolds (empty file)" and the output therefore missed scaffolds.fasta but there was configs.fasta. Any clue if it is a memory issue or anything else?

I use veresion 3.11.1 and set num_thread to 50 and memory limit to 1024 GB and run on a machine with 96 cores and 1.5TB RAM.

Any help is appreciated!

Hi,

I'm trying to run SPAdes with different number of threads (4, 8, 16, 32 and 64), but for 32 and 64 threads it stops abruptly. Is this expected?

This is my command:
spades.py --careful --only-assembler --threads 32 --memory 25 --cov-cutoff off -k 55,77,99 -1 /home/msilva/campy_test/32_threads/ERR459548/ERR459548/trimmomatic/ERR459548_2P.fastq.gz -2 /home/msilva/campy_test/32_threads/ERR459548/ERR459548/trimmomatic/ERR459548_1P.fastq.gz -o /home/msilva/campy_test/32_threads/ERR459548/ERR459548/spades/

In attachment you can find the STDOUT (stdout.txt) and STDERR (stderr.txt).

Thank you in advance.
Best regards,
Miguel

Hello,

I will be using spades to assemble reads de-novo. I have aligned my set of reads to a genome and then I have extracted those reads which failed to align. I have decided to assemble those reads de-novo using spades. For those reads which do map to the genome I am using some alignment driven assembler. I have 5 libraries to assemble. I am not sure what value I should use for '-k' and '-cov-cutoff'. I have attached a histogram depicting read length distribution.

Thank you.