sanger-pathogens / circlator Goto Github PK

A tool to circularize genome assemblies

Home Page: http://sanger-pathogens.github.io/circlator/

License: Other

Python 96.94% Shell 1.96% Dockerfile 1.10%

bioinformatics bioinformatics-pipeline genomics global-health infectious-diseases next-generation-sequencing pathogen research sequencing

circlator's People

Contributors

Stargazers

Watchers

Forkers

martinghunt andrewjpage xtmgah aslett1 crashfrog komwit gedankenstuecke wenchaolin bretonics prasanjoglekar nicoladm spock icedevil2001 zhangrengang frebio mictadlo pythseq thackl mptrsen pathogen-informatics ybendana gavieira pjeraldo srooke touala zhanmengtao tmsincomb microdm ypchan jungmin510 germant13 zhutao1009 krol33 sequana vittorio-a hyphaltip zhaoxia413 ayixon jialonghao1997 genostack sumsarium matt-plumb upalabdhad hieuvodl cautree mr-c fallinwind danpal96 cahyaprihatna randolium odielwoolmore ramongallego

circlator's Issues

progcheck: found nucmer but couldn't get version

Having installed circlator using pip3 install then added it to my PATH, I tried circlator progcheck. All the dependencies are there except one:

ubuntu@nat:~/Data/BP_UK36/gDNA/canu$ circlator progcheck
Circlator version: 1.5.1

External dependencies:
bwa     0.7.15  /home/ubuntu/biobuilds-2017.05/bin/bwa
canu    1.5     /home/ubuntu/biobuilds-2017.05/bin/canu
**Found nucmer but couldn't get version.**
prodigal        2.6.3   /home/ubuntu/biobuilds-2017.05/bin/prodigal
samtools        1.3.1   /home/ubuntu/biobuilds-2017.05/bin/samtools
WARNING: SPAdes version 3.10.1 is being used. It will work, but better results are usually obtained from Circlator using SPAdes version 3.7.1. Although 3.7.1 is not the latest version, we recommend it for Circlator.
spades  3.10.1  /home/ubuntu/biobuilds-2017.05/bin/spades.py

Python version:
3.5.2 (default, Nov 17 2016, 17:05:23)
[GCC 5.4.0 20160609]

Python dependencies:
openpyxl        2.4.8   /home/ubuntu/.local/lib/python3.5/site-packages/openpyxl/__init__.py
pyfastaq        3.15.0  /home/ubuntu/.local/lib/python3.5/site-packages/pyfastaq/__init__.py
pymummer        0.10.2  /home/ubuntu/.local/lib/python3.5/site-packages/pymummer/__init__.py
pysam   0.11.2.2        /home/ubuntu/.local/lib/python3.5/site-packages/pysam/__init__.py

Any ideas as to how can I fix this, please? Why would nucmer's version not be findable?

Thank you,
Nat

Installation problem: command not found after pip3 install circlator

Hi,

I checked all the dependencies and they're all up-to-date and in my PATH, then I used pip3 install circlator to install circlator, and it seemed to install fine:

ubuntu@nat_CLIMB:~$ pip3 install circlator
Collecting circlator
Collecting openpyxl (from circlator)
Collecting pymummer>=0.9.0 (from circlator)
Collecting pysam>=0.8.1 (from circlator)
  Using cached pysam-0.11.2.2-cp35-cp35m-manylinux1_x86_64.whl
Collecting pyfastaq>=3.12.1 (from circlator)
Collecting et-xmlfile (from openpyxl->circlator)
Collecting jdcal (from openpyxl->circlator)
Installing collected packages: et-xmlfile, jdcal, openpyxl, pyfastaq, pymummer, pysam, circlator
Successfully installed circlator-1.5.1 et-xmlfile-1.0.1 jdcal-1.3 openpyxl-2.4.8 pyfastaq-3.15.0 pymummer-0.10.2 pysam-0.11.2.2
You are using pip version 8.1.1, however version 9.0.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.

But when I type $ circlator I get circlator: command not found in response (same if I do circlator all or circlator progcheck).

Do I need to add circlator to my PATH manually and, if so, where will pip3 install have installed circlator please? Or is this a different problem entirely? I'm using python 3.5.2 on ubuntu 16.04 if that's relevant at all!

Thank you,
Nat

Use "--threads" option for samtools view

Latests versions of samtools now support multithreading. Code should be updated accordingly (will require a version check).

Recompile with -fPIC error git / pip3 installs

Dear devs,

I'm trying to install circlator 1.5.1 on my machine but I keep getting the same error either with pip3 install or git clone then python3 setup.py test.

Any idea on how to fix that ? I'm running Bash on Windows 10 and will try it as soon as I can on another machine to see if that's the issue.

Here is (the end of) the output error:

# pysam: htslib_config LIBS=-llzma -lbz2 -lz -lm -lcurl -lcrypto
# pysam: config_option: ENABLE_PLUGINS=0
# pysam: config_option: HAVE_COMMONCRYPTO=0
# pysam: config_option: HAVE_GMTIME_R=1
# pysam: config_option: HAVE_HMAC=1
# pysam: config_option: HAVE_IRODS=0
# pysam: config_option: HAVE_LIBCURL=1
# pysam: config_option: HAVE_MMAP=1
package init file 'samtools/__init__.py' not found (or not a regular file)
package init file 'bcftools/__init__.py' not found (or not a regular file)
package init file 'samtools/win32/__init__.py' not found (or not a regular file)
package init file 'htslib/__init__.py' not found (or not a regular file)
package init file 'htslib/htslib/__init__.py' not found (or not a regular file)
warning: no files found matching 'KNOWN_BUGS'
warning: no files found matching 'THANKS'
warning: no files found matching 'samtools/configure'
warning: no files found matching 'samtools/config.mk.in'
warning: no files found matching 'samtools/config.h.in'
warning: no files found matching 'tests/pysam_data'
warning: no files found matching 'tests/tabix_data'
/usr/bin/ld: //usr/local/lib/libbz2.a(bzlib.o): relocation R_X86_64_32S against `BZ2_crc32Table' can not be used when making a shared object; recompile with -fPIC
//usr/local/lib/libbz2.a: error adding symbols: Bad value
collect2: error: ld returned 1 exit status
error: Setup script exited with error: command 'x86_64-linux-gnu-gcc' failed with exit status 1

Start fixer too stringent on hit length and query length

Line 144 of start_fixer is too stringent with its first condition:

if match.hit_length_qry == match.qry_length 
    and match.percent_identity >= min_percent_id:  
        if original_contig_name not in circularized 
            or circularized[original_contig_name].percent_identity < match.percent_identity:
                    circularized[original_contig_name] = copy.copy(match)

I propose something like this instead?

# Divide the smaller of hit and qry lengths by the larger of hit and qry lengths
match_hit_qry_ratio_as_percent = 100 * min(match.hit_length_qry, match.qry_length) / max(match.hit_length_qry, match.qry_length)
# Ensure that the percentage matching and the length ratios are greater than the min_percent_ids.
if match_hit_qry_ratio_as_percent >= min_percent_id 
    and match.percent_identity >= min_percent_id:
       if original_contig_name not in circularized 
            or circularized[original_contig_name].percent_identity < match.percent_identity: `

use of wget may break circlator

Hello,

circlator/dnaa.py use wget to get sequences from uniprot.

wget is not available on most linux distro by default, curl seems more common.

using built in python download mechanism will be safer.

regards

Eric

ImportError: No module named 'circlator'

Hi,

I tried installing Circlator and I'm getting stuck at this error. The test and install steps finish without any errors. However, when I try to run the prgcheck or circlator all commands, I get this error below -

[cdesai@endeavor ~]$ circlator 
Usage: circlator <command> [options] <required arguments>

To get minimal usage for a command use:
circlator command

To get full help for a command use one of:
circlator command -h
circlator command --help


Available commands:

all        Run mapreads, bam2reads, assemble, merge, clean, fixstart
mapreads   Map reads to assembly
bam2reads  Make reads from mapping to be reassembled
assemble   Run assembly using reads from bam2reads
merge      Merge original assembly and new assembly made by assemble
clean      Remove small and completely contained contigs from assembly
fixstart   Change start position of circular sequences
minimus2   Run the minimus2 based circularisation pipeline
get_dnaa   Download file of dnaA (or other of user's choice) genes
progcheck  Checks dependencies are installed
version    Print version and exit

[cdesai@endeavor ~]$ circlator all -h
Traceback (most recent call last):
  File "/usr/local/bin/circlator", line 4, in <module>
    __import__('pkg_resources').run_script('circlator==1.1.4', 'circlator')
  File "/usr/local/lib/python3.4/site-packages/pkg_resources/__init__.py", line 696, in run_script
    self.require(requires)[0].run_script(script_name, ns)
  File "/usr/local/lib/python3.4/site-packages/pkg_resources/__init__.py", line 1621, in run_script
    exec(script_code, namespace, namespace)
  File "/usr/local/lib/python3.4/site-packages/circlator-1.1.4-py3.4.egg/EGG-INFO/scripts/circlator", line 55, in <module>
  File "<string>", line 1, in <module>
ImportError: No module named 'circlator'

Could you tell me where this is going wrong?

Thank you,
Chandni

Circlator with short reads?

Hello,
I'm trying to verify a circularized genome. I have a single contig with what looks like overlapping ends, and I'd like to try and use circlator to finish the process, but all options appear to require long reads. I've tried doing the bwa mapping manually, but when I try to move on to the other steps, I get a series of errors. Before I go any further, can you tell me if there are any tricks to get circlator to run with short reads?

Thank you.

Install failed - bio-assembly-refinement/setup.py", line 6,

Any ideas?

tseemann@marvin:~ $ pip3 install circlator
You are using pip version 6.1.1, however version 7.0.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
Collecting circlator
  Downloading circlator-0.12.1.tar.gz (80kB)
    100% |████████████████████████████████| 81kB 1.2MB/s
Collecting nose>=1.3 (from circlator)
  Downloading nose-1.3.6-py3-none-any.whl (154kB)
    100% |████████████████████████████████| 155kB 1.8MB/s
Collecting openpyxl (from circlator)
  Downloading openpyxl-2.2.3.tar.gz (108kB)
    100% |████████████████████████████████| 110kB 1.3MB/s
Collecting pyfastaq>=3.5.0 (from circlator)
  Downloading pyfastaq-3.5.0.tar.gz (47kB)
    100% |████████████████████████████████| 49kB 2.9MB/s
Requirement already satisfied (use --upgrade to upgrade): pysam>=0.8.1 in /bio/linuxbrew/lib/python3.4/site-packages (from circlator)
Collecting pymummer>=0.2.0 (from circlator)
  Downloading pymummer-0.3.0.tar.gz
Collecting bio-assembly-refinement>=0.1.0 (from circlator)
  Downloading bio_assembly_refinement-0.3.0.tar.gz
    Complete output from command python setup.py egg_info:
    Traceback (most recent call last):
      File "<string>", line 20, in <module>
      File "/tmp/pip-build-uj4icwuj/bio-assembly-refinement/setup.py", line 12, in <module>
        long_description=read('README.md'),
      File "/tmp/pip-build-uj4icwuj/bio-assembly-refinement/setup.py", line 6, in read
        return open(os.path.join(os.path.dirname(__file__), fname)).read()
    FileNotFoundError: [Errno 2] No such file or directory: '/tmp/pip-build-uj4icwuj/bio-assembly-refinement/README.md'

    ----------------------------------------
    Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-build-uj4icwuj/bio-assembly-refinement

Error running circlator with option --assemble_not_only_assembler

I ran circlator with option --assemble_not_only_assembler and got an error:

svkazakov@svkazakov /media/hdd/genome/assembly/B.pumilus_7P/new2/plasmid % circlator all --threads 6 --verbose --merge_min_length 100 --merge_min_length_merge 200 --assemble_not_only_assembler sp454Ion/scaffolds.fasta ../7P.tr.1t.fastq circSp454Ion-4
_________________________ Checking external programs __________________________
Circlator version: 1.4.0

External dependencies:
bwa 0.7.12 /usr/bin/bwa
nucmer 3.1 /usr/bin/nucmer
prodigal 2.6.2 /usr/bin/prodigal
samtools 0.1.19 /usr/bin/samtools
spades 3.7.1 /opt/SPAdes-3.7.1/bin/spades.py

Python version:
3.5.2 (default, Nov 17 2016, 17:05:23)
[GCC 5.4.0 20160609]

Python dependencies:
openpyxl 2.4.1 /home/svkazakov/.local/lib/python3.5/site-packages/openpyxl/init.py
pyfastaq 3.14.0 /home/svkazakov/.local/lib/python3.5/site-packages/pyfastaq/init.py
pymummer 0.10.1 /home/svkazakov/.local/lib/python3.5/site-packages/pymummer/init.py
pysam 0.9.1.4 /home/svkazakov/.local/lib/python3.5/site-packages/pysam/init.py
______________________________ Running mapreads _______________________________
samtools 0.1.19 /usr/bin/samtools
bwa 0.7.12 /usr/bin/bwa
syscall: bwa index -p 01.mapreads.bam.tmp.bwa_index 00.input_assembly.fasta
syscall: bwa mem -x pacbio -t 6 01.mapreads.bam.tmp.bwa_index /media/hdd/genome/assembly/B.pumilus_7P/new2/7P.tr.1t.fastq | samtools view -F 0x0800 -T 00.input_assembly.fasta -b -o 01.mapreads.bam.tmp.unsorted.bam -
syscall: samtools sort -@ 4 -m 125M 01.mapreads.bam.tmp.unsorted.bam 01.mapreads
syscall: samtools index 01.mapreads.bam
______________________________ Running bam2reads ______________________________
Getting reads from BAM file /media/hdd/genome/assembly/B.pumilus_7P/new2/plasmid/circSp454Ion-4/01.mapreads.bam
Getting all reads that map to contig NODE_10_length_384_cov_25.0608_component_0
...
Getting all unmapped reads
Finished getting reads.
Log file: /media/hdd/genome/assembly/B.pumilus_7P/new2/plasmid/circSp454Ion-4/02.bam2reads.log
Reads file: /media/hdd/genome/assembly/B.pumilus_7P/new2/plasmid/circSp454Ion-4/02.bam2reads.fasta
______________________________ Running assemble _______________________________
spades 3.7.1 /opt/SPAdes-3.7.1/bin/spades.py
samtools 0.1.19 /usr/bin/samtools
syscall: /opt/SPAdes-3.7.1/bin/spades.py -s /media/hdd/genome/assembly/B.pumilus_7P/new2/plasmid/circSp454Ion-4/02.bam2reads.fasta -o /media/hdd/genome/assembly/B.pumilus_7P/new2/plasmid/circSp454Ion-4/03.assemble.tmp.spades.127.17u586vh -t 6 -k 127 --careful
syscall: /opt/SPAdes-3.7.1/bin/spades.py -s /media/hdd/genome/assembly/B.pumilus_7P/new2/plasmid/circSp454Ion-4/02.bam2reads.fasta -o /media/hdd/genome/assembly/B.pumilus_7P/new2/plasmid/circSp454Ion-4/03.assemble.tmp.spades.117.gtltvt89 -t 6 -k 117 --careful
syscall: /opt/SPAdes-3.7.1/bin/spades.py -s /media/hdd/genome/assembly/B.pumilus_7P/new2/plasmid/circSp454Ion-4/02.bam2reads.fasta -o /media/hdd/genome/assembly/B.pumilus_7P/new2/plasmid/circSp454Ion-4/03.assemble.tmp.spades.107.y06swgpi -t 6 -k 107 --careful
syscall: /opt/SPAdes-3.7.1/bin/spades.py -s /media/hdd/genome/assembly/B.pumilus_7P/new2/plasmid/circSp454Ion-4/02.bam2reads.fasta -o /media/hdd/genome/assembly/B.pumilus_7P/new2/plasmid/circSp454Ion-4/03.assemble.tmp.spades.97.j0_iyhei -t 6 -k 97 --careful
syscall: /opt/SPAdes-3.7.1/bin/spades.py -s /media/hdd/genome/assembly/B.pumilus_7P/new2/plasmid/circSp454Ion-4/02.bam2reads.fasta -o /media/hdd/genome/assembly/B.pumilus_7P/new2/plasmid/circSp454Ion-4/03.assemble.tmp.spades.87.6eo41c9d -t 6 -k 87 --careful
syscall: /opt/SPAdes-3.7.1/bin/spades.py -s /media/hdd/genome/assembly/B.pumilus_7P/new2/plasmid/circSp454Ion-4/02.bam2reads.fasta -o /media/hdd/genome/assembly/B.pumilus_7P/new2/plasmid/circSp454Ion-4/03.assemble.tmp.spades.77._0rqlwyg -t 6 -k 77 --careful
Traceback (most recent call last):
File "/home/svkazakov/.local/bin/circlator", line 58, in
exec('circlator.tasks.' + task + '.run()')
File "", line 1, in
File "/home/svkazakov/.local/lib/python3.5/site-packages/circlator/tasks/all.py", line 161, in run
a.run()
File "/home/svkazakov/.local/lib/python3.5/site-packages/circlator/assemble.py", line 123, in run
self.run_spades(stop_at_first_success=self.spades_use_first_success)
File "/home/svkazakov/.local/lib/python3.5/site-packages/circlator/assemble.py", line 118, in run_spades
raise Error('Error running SPAdes. Output directories are:\n ' + '\n '.join(kmer_to_dir.values()) + '\nThe reason why should be in the spades.log file in each directory.')
circlator.assemble.Error: Error running SPAdes. Output directories are:
/media/hdd/genome/assembly/B.pumilus_7P/new2/plasmid/circSp454Ion-4/03.assemble.tmp.spades.97.j0_iyhei
/media/hdd/genome/assembly/B.pumilus_7P/new2/plasmid/circSp454Ion-4/03.assemble.tmp.spades.117.gtltvt89
/media/hdd/genome/assembly/B.pumilus_7P/new2/plasmid/circSp454Ion-4/03.assemble.tmp.spades.87.6eo41c9d
/media/hdd/genome/assembly/B.pumilus_7P/new2/plasmid/circSp454Ion-4/03.assemble.tmp.spades.107.y06swgpi
/media/hdd/genome/assembly/B.pumilus_7P/new2/plasmid/circSp454Ion-4/03.assemble.tmp.spades.77._0rqlwyg
/media/hdd/genome/assembly/B.pumilus_7P/new2/plasmid/circSp454Ion-4/03.assemble.tmp.spades.127.17u586vh
The reason why should be in the spades.log file in each directory.

Spades directories didn't contain any log files, but when i ran SPAdes by command specified above, i got:
== Error == to run read error correction, reads should be in FASTQ format (.fq, .fastq, .bam, .fq.gz, .fastq.gz, .bam.gz are supported): /media/hdd/genome/assembly/B.pumilus_7P/new2/plasmid/circSp454Ion-4/02.bam2reads.fasta (single reads, library number: 1, library type: single)

Spades version detection error

I just installed Circlator version: 1.2.1 using pip3 on my mac (OS X El Captain) after installing all the dependencies in the correct versions (I believe), but when I run progcheck, it gives me an error: Found spades but couldn't get version. Any suggestions to solve this issue? Thanks!

Below is the output when I type in "circlator progcheck".

Circlator version: 1.2.1

External dependencies:
bwa 0.7.15 xxxxxx
nucmer 3.1 xxxxxx
prodigal 2.6.3 xxxxxx
samtools 1.3.1 xxxxxx
Found spades but couldn't get version.

Python version:
3.5.2 (v3.5.2:4def2a2901a5, Jun 26 2016, 10:47:25)
[GCC 4.2.1 (Apple Inc. build 5666) (dot 3)]

Python dependencies:
bio_assembly_refinement 0.5.1 /Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/bio_assembly_refinement/init.py
openpyxl 2.3.5 /Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/openpyxl/init.py
pyfastaq 3.12.1 /Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/pyfastaq/init.py
pymummer 0.7.1 /Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/pymummer/init.py
pysam 0.8.3 /Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/pysam/init.py

Request: unify log file output

(7) log files are currently provided:

02.bam2reads.log
04.merge.circularise_details.log
04.merge.circularise.log
04.merge.merge.iterations.log
05.clean.log
06.fixstart.detailed.log
06.fixstart.log

All except 06.fixstart.detailed.log have the first column as [<ACTION>]. Could [fixstart details] be added as the first column?

Individual log files are useful, but a unified output format would enable a single log file to be saved for archiving, e.g., cat *.log | pigz -c > ../circ.log.gz

Numpy for python3

I am trying to install circlator on Ubuntu 14.04
I am having an issue with numpy for python3

When installing circlator using brew I get the following error

Error! numpy for Python3 not found. Please install it (e.g. apt-get install python3-numpy)

sudo apt-get install python3.2-numpy Reading package lists... Done Building dependency tree Reading state information... Done python3-numpy is already the newest version. 0 to upgrade, 0 to newly install, 0 to remove and 37 not to upgrade.

If I try to install circlator via pip I get a similar error

Downloading/unpacking circlator
  Downloading circlator-0.16.0.tar.gz (935kB): 935kB downloaded
  Running setup.py (path:/tmp/pip_build_brian/circlator/setup.py) egg_info for package circlator

Downloading/unpacking openpyxl (from circlator)
  Downloading openpyxl-2.2.5.tar.gz (108kB): 108kB downloaded
  Running setup.py (path:/tmp/pip_build_brian/openpyxl/setup.py) egg_info for package openpyxl

no previously-included directories found matching 'openpyxl/tests'
no previously-included directories found matching 'openpyxl/sample'
no previously-included directories found matching 'openpyxl/benchmarks'
no previously-included directories found matching 'openpyxl/classify'
warning: no previously-included files matching 'test_.py' found under directory 'openpyxl'
warning: no previously-included files matching 'tests/.py' found under directory 'openpyxl'


Downloading/unpacking pyfastaq>=3.6.0 (from circlator)
  Downloading pyfastaq-3.6.0.tar.gz (48kB): 48kB downloaded
  Running setup.py (path:/tmp/pip_build_brian/pyfastaq/setup.py) egg_info for package pyfastaq
    Traceback (most recent call last):
      File "<string>", line 17, in <module>
      File "/tmp/pip_build_brian/pyfastaq/setup.py", line 10
        print("Error! numpy for Python3 not found.\nPlease install it (e.g. apt-get install python3-numpy)", file=sys.stderr)
                                                                                                                 ^
    SyntaxError: invalid syntax
    Complete output from command python setup.py egg_info:
    Traceback (most recent call last):

  File "<string>", line 17, in <module>

  File "/tmp/pip_build_brian/pyfastaq/setup.py", line 10

print("Error! numpy for Python3 not found.\nPlease install it (e.g. apt-get install python3-numpy)", file=sys.stderr)


SyntaxError: invalid syntax

---

Cleaning up...
Command python setup.py egg_info failed with error code 1 in /tmp/pip_build_brian/pyfastaq
Storing debug log for failure in /tmp/tmpx5JulT ```

circlator.assembly.Error

Hi, there
I tested circlator with test command and it worked perfect. However, when I entered the following commands
circlator all --verbose --threads 8 --merge_min_id 85 --merge_breaklen 1000 96200minion_mc10.fasta minion_trimmedReads.fasta ./spades/
it stopped at somewhere near "Making ACT files from 04.merge.merge.iter.2.merged.fasta"

please see the detailed information below

_________________________ Checking external programs __________________________
Circlator version: 1.5.1

External dependencies:
bwa 0.7.15 /home/linuxbrew/.linuxbrew/bin/bwa
canu 1.5 /home/linuxbrew/.linuxbrew/bin/canu
nucmer 3.1 /home/linuxbrew/.linuxbrew/bin/nucmer
prodigal 2.6.2 /home/linuxbrew/.linuxbrew/bin/prodigal
samtools 1.4.1 /home/linuxbrew/.linuxbrew/bin/samtools
WARNING: SPAdes version 3.10.1 is being used. It will work, but better results are usually obtained from Circlator using SPAdes version 3.7.1. Although 3.7.1 is not the latest version, we recommend it for Circlator.
spades 3.10.1 /home/linuxbrew/.linuxbrew/bin/spades.py

Python version:
3.6.1 (default, Apr 8 2017, 07:19:26)
[GCC 4.8.4]

Python dependencies:
openpyxl 2.4.1 /home/linuxbrew/.linuxbrew/Cellar/circlator/1.5.1/libexec/lib/python3.6/site-packages/openpyxl/init.py
pyfastaq 3.14.0 /home/linuxbrew/.linuxbrew/Cellar/circlator/1.5.1/libexec/lib/python3.6/site-packages/pyfastaq/init.py
pymummer 0.10.1 /home/linuxbrew/.linuxbrew/Cellar/circlator/1.5.1/libexec/lib/python3.6/site-packages/pymummer/init.py
pysam 0.8.4 /home/linuxbrew/.linuxbrew/Cellar/circlator/1.5.1/libexec/lib/python3.6/site-packages/pysam/init.py
WARNING: SPAdes version 3.10.1 is being used. It will work, but better results are usually obtained from Circlator using SPAdes version 3.7.1. Although 3.7.1 is not the latest version, we recommend it for Circlator.
______________________________ Running mapreads _______________________________
samtools 1.4.1 /home/linuxbrew/.linuxbrew/bin/samtools
bwa 0.7.15 /home/linuxbrew/.linuxbrew/bin/bwa
syscall: bwa index -p 01.mapreads.bam.tmp.bwa_index 00.input_assembly.fasta
syscall: bwa mem -x pacbio -t 8 01.mapreads.bam.tmp.bwa_index /home/zong/Desktop/666/96200/circlator/minion_trimmedReads.fasta | samtools view -F 0x0800 -T 00.input_assembly.fasta -b -o 01.mapreads.bam.tmp.unsorted.bam -
syscall: samtools sort -@ 4 -m 125M 01.mapreads.bam.tmp.unsorted.bam -o 01.mapreads.bam
syscall: samtools index 01.mapreads.bam
______________________________ Running bam2reads ______________________________
Getting reads from BAM file /home/zong/Desktop/666/96200/circlator/spades/01.mapreads.bam
Getting all reads that map to ends of contig 96200minion_mc10_1. Coords: 1-50000;5022640-5072639
Getting all reads that map to contig 96200minion_mc10_10
Getting all reads that map to contig 96200minion_mc10_11
Getting all reads that map to contig 96200minion_mc10_12
Getting all reads that map to contig 96200minion_mc10_13
Getting all reads that map to contig 96200minion_mc10_14
Getting all reads that map to contig 96200minion_mc10_15
Getting all reads that map to contig 96200minion_mc10_16
Getting all reads that map to contig 96200minion_mc10_17
Getting all reads that map to contig 96200minion_mc10_18
Getting all reads that map to contig 96200minion_mc10_19
Getting all reads that map to ends of contig 96200minion_mc10_2. Coords: 1-50000;113721-163720
Getting all reads that map to contig 96200minion_mc10_20
Getting all reads that map to contig 96200minion_mc10_21
Getting all reads that map to contig 96200minion_mc10_22
Getting all reads that map to contig 96200minion_mc10_23
Getting all reads that map to contig 96200minion_mc10_24
Getting all reads that map to contig 96200minion_mc10_25
Getting all reads that map to contig 96200minion_mc10_26
Getting all reads that map to contig 96200minion_mc10_27
Getting all reads that map to contig 96200minion_mc10_28
Getting all reads that map to contig 96200minion_mc10_29
Getting all reads that map to ends of contig 96200minion_mc10_3. Coords: 1-50000;54749-104748
Getting all reads that map to contig 96200minion_mc10_30
Getting all reads that map to contig 96200minion_mc10_31
Getting all reads that map to contig 96200minion_mc10_32
Getting all reads that map to contig 96200minion_mc10_33
Getting all reads that map to contig 96200minion_mc10_34
Getting all reads that map to contig 96200minion_mc10_35
Getting all reads that map to contig 96200minion_mc10_36
Getting all reads that map to contig 96200minion_mc10_37
Getting all reads that map to contig 96200minion_mc10_38
Getting all reads that map to contig 96200minion_mc10_4
Getting all reads that map to contig 96200minion_mc10_5
Getting all reads that map to contig 96200minion_mc10_6
Getting all reads that map to contig 96200minion_mc10_7
Getting all reads that map to contig 96200minion_mc10_8
Getting all reads that map to contig 96200minion_mc10_9
Getting all unmapped reads
Finished getting reads.
Log file: /home/zong/Desktop/666/96200/circlator/spades/02.bam2reads.log
Reads file: /home/zong/Desktop/666/96200/circlator/spades/02.bam2reads.fasta
______________________________ Running assemble _______________________________
samtools 1.4.1 /home/linuxbrew/.linuxbrew/bin/samtools
WARNING: SPAdes version 3.10.1 is being used. It will work, but better results are usually obtained from Circlator using SPAdes version 3.7.1. Although 3.7.1 is not the latest version, we recommend it for Circlator.
spades 3.10.1 /home/linuxbrew/.linuxbrew/bin/spades.py
syscall: spades.py -s /home/zong/Desktop/666/96200/circlator/spades/02.bam2reads.fasta -o /home/zong/Desktop/666/96200/circlator/spades/03.assemble.tmp.spades.127.zja_p98v -t 8 -k 127 --careful --only-assembler
syscall: samtools faidx /home/zong/Desktop/666/96200/circlator/spades/03.assemble.tmp.spades.127.zja_p98v/contigs.fasta
syscall: spades.py -s /home/zong/Desktop/666/96200/circlator/spades/02.bam2reads.fasta -o /home/zong/Desktop/666/96200/circlator/spades/03.assemble.tmp.spades.117.zl1gtppk -t 8 -k 117 --careful --only-assembler
syscall: samtools faidx /home/zong/Desktop/666/96200/circlator/spades/03.assemble.tmp.spades.117.zl1gtppk/contigs.fasta
syscall: spades.py -s /home/zong/Desktop/666/96200/circlator/spades/02.bam2reads.fasta -o /home/zong/Desktop/666/96200/circlator/spades/03.assemble.tmp.spades.107.t_iaspce -t 8 -k 107 --careful --only-assembler
syscall: samtools faidx /home/zong/Desktop/666/96200/circlator/spades/03.assemble.tmp.spades.107.t_iaspce/contigs.fasta
syscall: spades.py -s /home/zong/Desktop/666/96200/circlator/spades/02.bam2reads.fasta -o /home/zong/Desktop/666/96200/circlator/spades/03.assemble.tmp.spades.97.4_cfct66 -t 8 -k 97 --careful --only-assembler
syscall: samtools faidx /home/zong/Desktop/666/96200/circlator/spades/03.assemble.tmp.spades.97.4_cfct66/contigs.fasta
syscall: spades.py -s /home/zong/Desktop/666/96200/circlator/spades/02.bam2reads.fasta -o /home/zong/Desktop/666/96200/circlator/spades/03.assemble.tmp.spades.87.wuyfga3j -t 8 -k 87 --careful --only-assembler
syscall: samtools faidx /home/zong/Desktop/666/96200/circlator/spades/03.assemble.tmp.spades.87.wuyfga3j/contigs.fasta
syscall: spades.py -s /home/zong/Desktop/666/96200/circlator/spades/02.bam2reads.fasta -o /home/zong/Desktop/666/96200/circlator/spades/03.assemble.tmp.spades.77.1_sat3zg -t 8 -k 77 --careful --only-assembler
syscall: samtools faidx /home/zong/Desktop/666/96200/circlator/spades/03.assemble.tmp.spades.77.1_sat3zg/contigs.fasta
[assemble] kmer N50
[assemble] 77 53776
[assemble] 87 53792
[assemble] 97 37763
[assemble] 107 39658
[assemble] 117 44019
[assemble] 127 53758
[assemble] using assembly with kmer 87
________________________________ Running merge ________________________________
Running command: bash run_nucmer.sh
1: PREPARING DATA
2,3: RUNNING mummer AND CREATING CLUSTERS

reading input file "p.ntref" of length 6680340

construct suffix tree for sequence of length 6680340

(maximum reference length is 536870908)

(maximum query length is 4294967295)

process 66803 characters per dot

#....................................................................................................

CONSTRUCTIONTIME /home/linuxbrew/.linuxbrew/Cellar/mummer/3.23_1/mummer p.ntref 2.84

reading input file "/home/zong/Desktop/666/96200/circlator/spades/03.assemble/contigs.fasta" of length 1067392

matching query-file "/home/zong/Desktop/666/96200/circlator/spades/03.assemble/contigs.fasta"

against subject-file "p.ntref"

COMPLETETIME /home/linuxbrew/.linuxbrew/Cellar/mummer/3.23_1/mummer p.ntref 3.81

SPACE /home/linuxbrew/.linuxbrew/Cellar/mummer/3.23_1/mummer p.ntref 7.51

4: FINISHING DATA

Making ACT files from 00.input_assembly.fasta /home/zong/Desktop/666/96200/circlator/spades/03.assemble/contigs.fasta 04.merge.merge.iter.1.coords
samtools 1.4.1 /home/linuxbrew/.linuxbrew/bin/samtools
bwa 0.7.15 /home/linuxbrew/.linuxbrew/bin/bwa
syscall: bwa index -p 04.merge.merge.iter.2.bam.tmp.bwa_index 04.merge.merge.iter.2.merged.fasta
syscall: bwa mem -x pacbio -t 8 04.merge.merge.iter.2.bam.tmp.bwa_index 02.bam2reads.fasta | samtools view -F 0x0800 -T 04.merge.merge.iter.2.merged.fasta -b -o 04.merge.merge.iter.2.bam.tmp.unsorted.bam -
syscall: samtools sort -@ 4 -m 125M 04.merge.merge.iter.2.bam.tmp.unsorted.bam -o 04.merge.merge.iter.2.bam
syscall: samtools index 04.merge.merge.iter.2.bam
samtools 1.4.1 /home/linuxbrew/.linuxbrew/bin/samtools
WARNING: SPAdes version 3.10.1 is being used. It will work, but better results are usually obtained from Circlator using SPAdes version 3.7.1. Although 3.7.1 is not the latest version, we recommend it for Circlator.
spades 3.10.1 /home/linuxbrew/.linuxbrew/bin/spades.py
syscall: spades.py -s /home/zong/Desktop/666/96200/circlator/spades/04.merge.merge.iter.2.reads.fasta -o /home/zong/Desktop/666/96200/circlator/spades/04.merge.merge.iter.2.assembly.tmp.spades.127.4ujuu8og -t 8 -k 127 --careful --only-assembler
syscall: samtools faidx /home/zong/Desktop/666/96200/circlator/spades/04.merge.merge.iter.2.assembly.tmp.spades.127.4ujuu8og/contigs.fasta
syscall: spades.py -s /home/zong/Desktop/666/96200/circlator/spades/04.merge.merge.iter.2.reads.fasta -o /home/zong/Desktop/666/96200/circlator/spades/04.merge.merge.iter.2.assembly.tmp.spades.117.xprip4ey -t 8 -k 117 --careful --only-assembler
syscall: samtools faidx /home/zong/Desktop/666/96200/circlator/spades/04.merge.merge.iter.2.assembly.tmp.spades.117.xprip4ey/contigs.fasta
syscall: spades.py -s /home/zong/Desktop/666/96200/circlator/spades/04.merge.merge.iter.2.reads.fasta -o /home/zong/Desktop/666/96200/circlator/spades/04.merge.merge.iter.2.assembly.tmp.spades.107.nfcjenuc -t 8 -k 107 --careful --only-assembler
syscall: samtools faidx /home/zong/Desktop/666/96200/circlator/spades/04.merge.merge.iter.2.assembly.tmp.spades.107.nfcjenuc/contigs.fasta
syscall: spades.py -s /home/zong/Desktop/666/96200/circlator/spades/04.merge.merge.iter.2.reads.fasta -o /home/zong/Desktop/666/96200/circlator/spades/04.merge.merge.iter.2.assembly.tmp.spades.97.8ormd5qy -t 8 -k 97 --careful --only-assembler
syscall: samtools faidx /home/zong/Desktop/666/96200/circlator/spades/04.merge.merge.iter.2.assembly.tmp.spades.97.8ormd5qy/contigs.fasta
syscall: spades.py -s /home/zong/Desktop/666/96200/circlator/spades/04.merge.merge.iter.2.reads.fasta -o /home/zong/Desktop/666/96200/circlator/spades/04.merge.merge.iter.2.assembly.tmp.spades.87.v4phbqci -t 8 -k 87 --careful --only-assembler
syscall: samtools faidx /home/zong/Desktop/666/96200/circlator/spades/04.merge.merge.iter.2.assembly.tmp.spades.87.v4phbqci/contigs.fasta
syscall: spades.py -s /home/zong/Desktop/666/96200/circlator/spades/04.merge.merge.iter.2.reads.fasta -o /home/zong/Desktop/666/96200/circlator/spades/04.merge.merge.iter.2.assembly.tmp.spades.77.ljnbbsrb -t 8 -k 77 --careful --only-assembler
syscall: samtools faidx /home/zong/Desktop/666/96200/circlator/spades/04.merge.merge.iter.2.assembly.tmp.spades.77.ljnbbsrb/contigs.fasta
[assemble] kmer N50
[assemble] 77 55319
[assemble] 87 55381
[assemble] 97 37763
[assemble] 107 39658
[assemble] 117 53751
[assemble] 127 53758
[assemble] using assembly with kmer 87
Running command: bash run_nucmer.sh
1: PREPARING DATA
2,3: RUNNING mummer AND CREATING CLUSTERS

reading input file "p.ntref" of length 6669533

construct suffix tree for sequence of length 6669533

(maximum reference length is 536870908)

(maximum query length is 4294967295)

process 66695 characters per dot

#....................................................................................................

CONSTRUCTIONTIME /home/linuxbrew/.linuxbrew/Cellar/mummer/3.23_1/mummer p.ntref 2.69

reading input file "/home/zong/Desktop/666/96200/circlator/spades/04.merge.merge.iter.2.assembly/contigs.fasta" of length 1188473

matching query-file "/home/zong/Desktop/666/96200/circlator/spades/04.merge.merge.iter.2.assembly/contigs.fasta"

against subject-file "p.ntref"

COMPLETETIME /home/linuxbrew/.linuxbrew/Cellar/mummer/3.23_1/mummer p.ntref 3.77

SPACE /home/linuxbrew/.linuxbrew/Cellar/mummer/3.23_1/mummer p.ntref 7.62

4: FINISHING DATA

Making ACT files from 04.merge.merge.iter.2.merged.fasta /home/zong/Desktop/666/96200/circlator/spades/04.merge.merge.iter.2.assembly/contigs.fasta 04.merge.merge.iter.2.coords
Traceback (most recent call last):
File "/home/linuxbrew/.linuxbrew/bin/circlator", line 58, in
exec('circlator.tasks.' + task + '.run()')
File "", line 1, in
File "/home/linuxbrew/.linuxbrew/Cellar/circlator/1.5.1/libexec/lib/python3.6/site-packages/circlator/tasks/all.py", line 212, in run
m.run()
File "/home/linuxbrew/.linuxbrew/Cellar/circlator/1.5.1/libexec/lib/python3.6/site-packages/circlator/merge.py", line 825, in run
self._circularise_contigs(nucmer_hits)
File "/home/linuxbrew/.linuxbrew/Cellar/circlator/1.5.1/libexec/lib/python3.6/site-packages/circlator/merge.py", line 351, in _circularise_contigs
called_as_circular_by_spades = self.reassembly.circular_contigs()
File "/home/linuxbrew/.linuxbrew/Cellar/circlator/1.5.1/libexec/lib/python3.6/site-packages/circlator/assembly.py", line 183, in circular_contigs
return self._circular_contigs_from_spades_after_3_6_1(self.assembly_graph_fastg, self.contigs_paths)
File "/home/linuxbrew/.linuxbrew/Cellar/circlator/1.5.1/libexec/lib/python3.6/site-packages/circlator/assembly.py", line 129, in _circular_contigs_from_spades_after_3_6_1
paths_dict = Assembly._spades_contigs_paths_to_dict(contigs_paths)
File "/home/linuxbrew/.linuxbrew/Cellar/circlator/1.5.1/libexec/lib/python3.6/site-packages/circlator/assembly.py", line 106, in _spades_contigs_paths_to_dict
raise Error('Error loading info from SPAdes contigs path file ' + filename)
circlator.assembly.Error: Error loading info from SPAdes contigs path file /home/zong/Desktop/666/96200/circlator/spades/04.merge.merge.iter.2.assembly/contigs.paths

Could someone help me to solve this? Thanks so much.

Circlator version: 1.5.1

Python version:
3.6.1 (default, Apr 8 2017, 07:19:26)
[GCC 4.8.4]

Canu version is not being detected properly

Seems to be regular expressions with v etc?

% canu -version
Canu 1.6
# this is printed to stdout, not stderr

% circlator  progcheck
Circlator version: 1.5.1

External dependencies:
bwa	0.7.16	/usr/local/bin/bwa
Found canu but couldn't get version.
nucmer	3.1	/usr/local/bin/nucmer
prodigal	2.6.3	/usr/local/bin/prodigal
samtools	1.5	/usr/local/bin/samtools
WARNING: SPAdes version 3.11.0 is being used. It will work, but better results are usually obtained from Circlator using SPAdes version 3.7.1. Although 3.7.1 is not the latest version, we recommend it for Circlator.
spades	3.11.0	/usr/local/bin/spades.py

Python version:
3.6.2 (default, Jul 17 2017, 16:44:45)
[GCC 4.2.1 Compatible Apple LLVM 8.1.0 (clang-802.0.42)]

Python dependencies:
openpyxl	2.4.1	/usr/local/Cellar/circlator/1.5.1/libexec/lib/python3.6/site-packages/openpyxl/__init__.py
pyfastaq	3.14.0	/usr/local/Cellar/circlator/1.5.1/libexec/lib/python3.6/site-packages/pyfastaq/__init__.py
pymummer	0.10.1	/usr/local/Cellar/circlator/1.5.1/libexec/lib/python3.6/site-packages/pymummer/__init__.py
pysam	0.8.4	/usr/local/Cellar/circlator/1.5.1/libexec/lib/python3.6/site-packages/pysam/__init__.py

Code regexp:

'canu': ('-version', re.compile('^Canu \D*([\d][\d\.]+)')),

Debug:

Running this command to get version: canu -version
__________ (begin output from canu -version)___________
Canu 1.6
__________ (end of output from canu -version)___________
Looking far a match to the regex "^Canu.*v.?([0-9][0-9\.]+)" in the above output
No match found to the regex
Found canu but couldn't get version.

Traceback (most recent call last):
  File "/home/linuxbrew/.linuxbrew/Cellar/circlator/1.5.1_1/bin/circlator", line 58, in <module>
    exec('circlator.tasks.' + task + '.run()')
  File "<string>", line 1, in <module>
  File "/home/linuxbrew/.linuxbrew/Cellar/circlator/1.5.1_1/libexec/lib/python3.6/site-packages/circlator/tasks/all.py", line 72, in run
    circlator.versions.get_all_versions(sys.stdout, raise_error=True, assembler=options.assembler)
  File "/home/linuxbrew/.linuxbrew/Cellar/circlator/1.5.1_1/libexec/lib/python3.6/site-packages/circlator/versions.py", line 15, in get_all_versions
    external_progs.check_all_progs(verbose=False, raise_error=raise_error, filehandle=filehandle, debug=debug, assembler=assembler)
  File "/home/linuxbrew/.linuxbrew/Cellar/circlator/1.5.1_1/libexec/lib/python3.6/site-packages/circlator/external_progs.py", line 108, in check_all_progs
    make_and_check_prog(prog, verbose=verbose, raise_error=raise_error, filehandle=filehandle, debug=debug, required=prog==assembler)
  File "/home/linuxbrew/.linuxbrew/Cellar/circlator/1.5.1_1/libexec/lib/python3.6/site-packages/circlator/external_progs.py", line 81, in make_and_check_prog
    handle_error('Found ' + name + " but couldn't get version.", raise_error=raise_error)
  File "/home/linuxbrew/.linuxbrew/Cellar/circlator/1.5.1_1/libexec/lib/python3.6/site-packages/circlator/external_progs.py", line 56, in handle_error
    raise Error(message + '\nCannot continue')
circlator.external_progs.Error: Found canu but couldn't get version.
Cannot continue

It looks like circlator keeps a common k-mer at the ends of a contig after changing start position. What for?

Hi,
I've run circlator on my own data, and it looks like it keeps a common k-mer at the ends of a contig, that was produced by SPAdes, after changing start position. Shouldn't circlator remove one of it?

So there was a common k-mer in a contig of input assembly, but the contig wasn't circular (see Bandage visualization in initial-assembly.png, + file 00.input_assembly.fasta)
After local assembly made by SPAdes 3.7.1 in task 03.assembly, the contig becomes circular (see after-local-assembly.png, + attached file 03.assemble-contigs.fasta). And now it has an common k-mer at the ends of it with size of 127 nucs, for used k=127. (see common-k-mer.png).
This contig NODE_1_* is then circularized using nucmer in task 04.merge but the file 04.merge.fasta still contains a common k-mer at the ends of the contig!
At the final output this contig is reversed and its start position is changed, but rc-copy of the common k-mer still presents two times in the middle of the final contig.

It is a bug, isn't it?

circlator.log.txt

other-files.zip

how to override the option "--ignore 06.fixstart.contigs_to_not_change" when running circlator all

Hello,

I have a circlator run for which I want to reset the start point using my own starting gene. When I just run with circlator fixstart, it worked as expected. However, when I run with circlator all, the start point was not reset. I noticed a file named 06.fixstart.contigs_to_not_change was generated this time and the contig matching with my specified starting gene is in this file. And according to this manual (https://github.com/sanger-pathogens/circlator/wiki/Task:-all), the option "--ignore 06.fixstart.contigs_to_not_change" is automatically activated by default, which should be responsible for my problem. I was wondering if there is a way to directly disable the option "--ignore 06.fixstart.contigs_to_not_change" when running circlator all? Or have I to run an extra step of circlator fixstart to get around?

Thanks in advance!

Best,
Jia-Xing

how to use circlator on computer without internet access

Hello,

one of our users reported a problem with circlator. running on our cluster.

NB our compute node are on a private network and DOES NOT have access to internet.
get_dnaa was failing due to no net access.

is there a way to provide circlator an alternate method to get dnaa ?
we already host this file on our data center and it can be accessed via plain file reading.

regards

Eric

fixstart: no suitable promer matches found (due to deletion in dnaA sequence of assembler output)

I've hit what's probably a rare corner case. Circularization completed successfully, but the start position of the chromosome was not set properly. I checked the logs for fixstart and this is what I see:

==> 06.fixstart.detailed.log <==
No suitable promer matches found
Using the following prodigal predictions to circularize contigs:
scf7180000000004|quiver Prodigal_v2.6.1 CDS 2205287 2207290 249.2   +   0   ID=1_2144;partial=00;start_type=ATG;rbs_motif=GGA/GAG/AGG;rbs_spacer=5-10bp;gc_cont=0.678;conf=99.99;score=249.24;cscore=242.85;sscore=6.39;rscore=1.96;uscore=0.46;tscore=3.96;

==> 06.fixstart.log <==
[fixstart]  id  break_point gene_name   gene_reversed   new_name    skipped
[fixstart]  scf7180000000004|quiver 2205287 prodigal    no  -   -

When I BLAST my organism's reference dnaA sequence against my assembly, I notice that my assembly's version is one base shorter. I suspect that there's a frameshift deletion in the assembly that mangles the translated protein sequence, preventing promer from making any matches. The deletion may be false and buff out during assembly polishing, but it still caused trouble here.

Problems running circlator

Hi,
I installed circlator from here (github) and after installing all the dependencies listed here I ran 'circlator progcheck' but this is the error I keep getting and I have tried a lot of things including reinstalling all the dependencies and nothing seems to work. Help please!

Circlator version: 1.5.1

External dependencies:
WARNING: Didn't find bwa in path. Looked for:bwa
WARNING: Didn't find canu in path. Looked for:canu
nucmer 3.1 /usr/bin/nucmer
WARNING: Didn't find prodigal in path. Looked for:prodigal
samtools 0.1.19 /usr/bin/samtools
WARNING: Didn't find spades in path. Looked for:spades.py

Python version:
3.5.2 (default, Nov 17 2016, 17:05:23)
[GCC 5.4.0 20160609]

Python dependencies:
openpyxl 2.4.8 /home/jojo/.local/lib/python3.5/site-packages/openpyxl/init.py
pyfastaq 3.15.0 /usr/local/lib/python3.5/dist-packages/pyfastaq-3.15.0-py3.5.egg/pyfastaq/init.py
pymummer 0.10.2 /usr/local/lib/python3.5/dist-packages/pymummer-0.10.2-py3.5.egg/pymummer/init.py
pysam 0.11.2.2 /home/jojo/.local/lib/python3.5/site-packages/pysam/init.py

Thanks in advance,

JoJo

Is it possible to customize canu's default setting in circlator?

Hello,

I was wondering if circlator can take customized canu parameters for the analysis? For example, I want to set "corMaxEvidenceErate=0.15" for canu when using circlator to reassemble the AT rich mitochondrial genome. Of course, I will select canu as the assembler for circlator in the same time. Thanks in advance!

Best,
Jia-Xing

memory usage

When I run circlator (circlator all) everything seems fine at the beginning, but after a couple of minutes running it crashes with this error message:

Traceback (most recent call last):
File "/home/ubuntu/.local/bin/circlator", line 58, in <module>
    exec('circlator.tasks.' + task + '.run()')
File "<string>", line 1, in <module>
File "/home/ubuntu/.local/lib/python3.5/site-packages/circlator/tasks/all.py", line 169, in run
    a.run()
File "/home/ubuntu/.local/lib/python3.5/site-packages/circlator/assemble.py", line 176, in run
    self.run_spades(stop_at_first_success=self.spades_use_first_success)
File "/home/ubuntu/.local/lib/python3.5/site-packages/circlator/assemble.py", line 145, in run_spades
    raise Error('Error running SPAdes. Output directories are:\n  ' + '\n  '.join(kmer_to_dir.values()) + '\nThe reason why should be in the spades.log file in each directory.')
circlator.assemble.Error: Error running SPAdes. Output directories are:
/home/ubuntu/Desktop/scripts/circlator-1.5.1/circlator/circlator_out/03.assemble.tmp.spades.97.85z_h7yn
/home/ubuntu/Desktop/scripts/circlator-1.5.1/circlator/circlator_out/03.assemble.tmp.spades.117.t9n_6clr
/home/ubuntu/Desktop/scripts/circlator-1.5.1/circlator/circlator_out/03.assemble.tmp.spades.87.79_qms10
/home/ubuntu/Desktop/scripts/circlator-1.5.1/circlator/circlator_out/03.assemble.tmp.spades.107.xsu1lo0e
/home/ubuntu/Desktop/scripts/circlator-1.5.1/circlator/circlator_out/03.assemble.tmp.spades.77.u2x2lpqh
/home/ubuntu/Desktop/scripts/circlator-1.5.1/circlator/circlator_out/03.assemble.tmp.spades.127.vaus39y9
The reason why should be in the spades.log file in each directory.

Going through the SPAdes log files, as advised in the error message, there seems to be a memory issue...


Command line: /home/ubuntu/Desktop/scripts/circlator-1.5.1/build/SPAdes-3.7.1-Linux/bin/spades.py -s /home/ubuntu/Desktop/scripts/circlator-1.5.1/circlator/circlator_out/02.bam2reads.fasta -o /home/ubuntu/Desktop/scripts/circlator-1.5.1/circlator/circlator_out/03.assemble.tmp.spades.77.u2x2lpqh -t 1 -k 77 --careful --only-assembler 

System information:
  SPAdes version: 3.7.1
  Python version: 2.7.12
  OS: Linux-4.4.0-83-generic-x86_64-with-Ubuntu-16.04-xenial

Output dir: /home/ubuntu/Desktop/scripts/circlator-1.5.1/circlator/circlator_out/03.assemble.tmp.spades.77.u2x2lpqh
Mode: ONLY assembling (without read error correction)
Debug mode is turned OFF

Dataset parameters:
  Multi-cell mode (you should set '--sc' flag if input data was obtained with MDA (single-cell) technology or --meta flag if processing metagenomic dataset)
  Reads:
    Library number: 1, library type: single
      left reads: not specified
      right reads: not specified
      interlaced reads: not specified
      single reads: ['/home/ubuntu/Desktop/scripts/circlator-1.5.1/circlator/circlator_out/02.bam2reads.fasta']
Assembly parameters:
  k: [77]
  Mismatch careful mode is turned ON
  Repeat resolution is enabled
  MismatchCorrector will be used
  Coverage cutoff is turned OFF
Other parameters:
  Dir for temp files: /home/ubuntu/Desktop/scripts/circlator-1.5.1/circlator/circlator_out/03.assemble.tmp.spades.77.u2x2lpqh/tmp
  Threads: 1
  Memory limit (in Gb): 1


======= SPAdes pipeline started. Log can be found here: /home/ubuntu/Desktop/scripts/circlator-1.5.1/circlator/circlator_out/03.assemble.tmp.spades.77.u2x2lpqh/spades.log


===== Assembling started.


== Running assembler: K77

  0:00:00.000     4M / 4M    INFO    General                 (memory_limit.hpp          :  43)   Memory limit set to 1 Gb
  0:00:00.004     4M / 4M    INFO    General                 (main.cpp                  : 148)   Starting SPAdes, built from refs/heads/spades_3.7.1, git revision 0a208242974e028693083a617d54e8dbc9e2577b
  0:00:00.005     4M / 4M    INFO    General                 (main.cpp                  : 149)   Assembling dataset (/home/ubuntu/Desktop/scripts/circlator-1.5.1/circlator/circlator_out/03.assemble.tmp.spades.77.u2x2lpqh/dataset.info) with K=77
  0:00:00.005     4M / 4M    INFO    General                 (launch.hpp                :  30)   SPAdes started
  0:00:00.005     4M / 4M    INFO    General                 (launch.hpp                :  36)   Starting from stage: construction
  0:00:00.005     4M / 4M    INFO    General                 (launch.hpp                :  39)   Two-step RR enabled: 0
  0:00:00.005     4M / 4M    INFO    General                 (launch.hpp                :  58)   Will need read mapping, kmer mapper will be attached
  0:00:00.006     4M / 4M    INFO   StageManager             (stage.cpp                 : 126)   STAGE == Construction
  0:00:00.006     4M / 4M    INFO    General                 (read_converter.hpp        :  80)   Converting reads to binary format (takes a while)
  0:00:00.008     4M / 4M    INFO    General                 (read_converter.hpp        :  87)   Paired reads for library #0
  0:00:00.959    68M / 132M  INFO    General                 (binary_converter.hpp      : 159)   0 reads written
  0:00:00.960     4M / 132M  INFO    General                 (read_converter.hpp        :  98)   Single reads for library #0
  0:00:02.610   144M / 260M  INFO    General                 (binary_converter.hpp      : 159)   1064 reads written
  0:00:02.613     4M / 260M  INFO    General                 (graph_construction.hpp    : 130)   Constructing DeBruijn graph for k=77
  0:00:02.613     4M / 260M  INFO    General                 (kmer_splitters.hpp        : 173)   Splitting kmer instances into 1 buckets. This might take a while.
  0:00:02.613     4M / 260M  INFO    General                 (file_limit.hpp            :  30)   Open file limit set to 1024
  0:00:02.613     4M / 260M  INFO    General                 (kmer_splitters.hpp        : 191)   Memory available for splitting buffers: 0.332031 Gb
  0:00:02.613     4M / 260M  INFO    General                 (kmer_splitters.hpp        : 200)   Using cell size of 14854826
  0:00:03.773   380M / 756M  INFO    General                 (kmer_splitters.hpp        : 235)   Adding contigs from previous K
  0:00:03.774   380M / 756M  INFO    General                 (kmer_splitters.hpp        : 246)   Used 2128 reads. Maximum read length 15105
  0:00:03.774   380M / 756M  INFO    General                 (kmer_splitters.hpp        : 247)   Average read length 4950.47
  0:00:03.775     4M / 756M  INFO    General                 (kmer_index.hpp            : 348)   Starting k-mer counting.
=== Stack Trace ===
[0x404fda]
[0x44e00c]
[0x472983]
[0x474945]
[0x44cc13]
[0x4781d0]
[0x498caa]
[0x4a3669]
[0x4a3caa]
[0x4a3d83]
[0x4a4057]
[0x4806a3]
[0x44a473]
[0x402502]
[0x417646]
[0x803650]
[0x4001b9]
Verification of expression '(intptr_t)MappedRegion != -1L' failed in function 'MMappedReader::MMappedReader(const string&, bool, size_t, off_t, size_t)'. In file '/home/yasha/gitrep3/algorithmic-biology/assembler/src/include/io/mmapped_reader.hpp' on line 97. Message 'mmap(2) failed. Reason: Cannot allocate memory. Error code: 12'.
Verification of expression '(intptr_t)MappedRegion != -1L' failed in function 'MMappedReader::MMappedReader(const string&, bool, size_t, off_t, size_t)'. In file '/home/yasha/gitrep3/algorithmic-biology/assembler/src/include/io/mmapped_reader.hpp' on line 97. Message 'mmap(2) failed. Reason: Cannot allocate memory. Error code: 12'.
spades: /home/yasha/gitrep3/algorithmic-biology/assembler/src/include/io/mmapped_reader.hpp:97: MMappedReader::MMappedReader(const string&, bool, size_t, off_t, size_t): Assertion `(intptr_t)MappedRegion != -1L' failed.


== Error ==  system call for: "['/home/ubuntu/Desktop/scripts/circlator-1.5.1/build/SPAdes-3.7.1-Linux/bin/spades', '/home/ubuntu/Desktop/scripts/circlator-1.5.1/circlator/circlator_out/03.assemble.tmp.spades.77.u2x2lpqh/K77/configs/config.info']" finished abnormally, err code: -6

In case you have troubles running SPAdes, you can write to [email protected]
Please provide us with params.txt and spades.log files from the output directory.

I work in a private company and my computing resources are quite limiting: I am not the administrator of my desktop PC, and I am forced to work in a virtual machine to work in a UNIX environment. The resources allocated to the virtual machine are 3Gb RAM and 2.2 Gb hard disk space free (the assembly and the corrected reads apart). In the circlator manuscript it is mentioned that the peak RAM usage was 490 Mb for a Salmonella genome but (I think) there is no mention to the hard disk storage memory required...
The bacterial genome I work with was sequenced with a PacBio RSII and assembled de novo with the RS_HGAP Assembly.2 pipeline. I got 10 contigs but I am not sure about the size of the genome (no reference is available for this species, and closest speceis vary between 4 and 6.5 Mb).
In summary, is 3 Gb RAM and 2.2 Gb hard disk space enough to run circlator to try and circularize 9-10 contigs from a baterial genome of 4-6 Mb?

circlator fails and still creates the final output file 06.fixstart.fasta

I've been testing circlator for use in an automated de novo assembly pipeline and see that it doesn't always work. Furthermore, it doesn't error out when something goes wrong, so my pipeline proceeds with assembly polishing despite the circularization not having been performed. Here are the logs for one failed sample:

==> 02.bam2reads.log <==
[bamfilter]     #contig length  reads_kept
[bamfilter]     scf7180000000004|quiver 4449764 1-50000;4399765-4449764

==> 04.merge.circularise_details.log <==
[merge circularise_details]     scf7180000000004|quiver Checking 3 nucmer hits
[merge circularise_details]     scf7180000000004|quiver potential pair of nucmer hits for circularization:
[merge circularise_details]     scf7180000000004|quiver         1       36542   36552   1       36542   36552   99.94   4449764 58223   1       scf7180000000004|quiver NODE_1_length_58223_cov_29.3807_ID_823
[merge circularise_details]     scf7180000000004|quiver         4406770 4440158 55874   22486   33389   33389   99.99   4449764 58223   1       scf7180000000004|quiver NODE_1_length_58223_cov_29.3807_ID_823
[merge circularise_details]     scf7180000000004|quiver cannot use this pair because positions/orientations of matches no good
[merge circularise_details]
[merge circularise_details]     SPAdes reassembly contigs that are circular: None
[merge circularise_details]
[merge circularise_details]     scf7180000000004|quiver Trying to circularize. Has nucmer hits to check...
[merge circularise_details]     scf7180000000004|quiver No matches to SPAdes circular contigs

==> 04.merge.circularise.log <==
[merge circularised]    #Contig repetitive_deleted      circl_using_nucmer      circl_using_spades      circularised
[merge circularised]    scf7180000000004|quiver 0       0       0       0

==> 04.merge.merge.iterations.log <==
[merge iterative_merge 1]       Using nucmer matches from 04.merge.merge.iter.1.coords
[merge iterative_merge 1]       You can view the nucmer matches with ACT using: ./04.merge.merge.iter.1.start_act.sh
[merge iterative_merge 1]       NODE_1_length_58223_cov_29.3807_ID_823: checking nucmer matches
[merge iterative_merge 1]       NODE_1_length_58223_cov_29.3807_ID_823: no potential pairs of hits to merge contigs
[merge iterative_merge 1]       Potential contigs to use for merging: 
[merge iterative_merge 1]       Contigs to use for merging after uniqueness filtering: 
[merge iterative_merge 1]       Made at least one contig join: False
[merge iterative_merge 1]       No contig merges were made

==> 05.clean.log <==
[clean] scf7180000000004|quiver kept

==> 06.fixstart.log <==
[fixstart]      id      break_point     gene_name       gene_reversed   new_name        skipped
[fixstart]      scf7180000000004|quiver -       -       -       -       skipped

It seems that the correct match was found, but circlator refused to use it. I ended up doing manual trimming and using minimus2 for this sample (This sample has a heterogeneous IS element insertion which caused this problem). I could work around this by adding code to my pipeline to check if the output file is the same as the input, but I think you should be aware of this problem. I was especially hoping that circlator would automatically resolve these troublesome cases.

Many thanks for this software.

Error running SPAdes.

What's wrong with the SPAdes or my input files ?

Here is the input files:
circlator all polished_assembly.fasta corrected.fasta circlator
The polished_assemble.fasta have 48 contigs ,range from 2k to 1.7M.

Here is the error report:
Traceback (most recent call last):
File "/pipeline/Circlator/vir_env/bin/circlator", line 5, in
pkg_resources.run_script('circlator==1.3.1', 'circlator')
File "/pipeline/Circlator/vir_env/lib/python3.5/site-packages/pkg_resources/init.py", line 746, in run_script
self.require(requires)[0].run_script(script_name, ns)
File "/pipeline/Circlator/vir_env/lib/python3.5/site-packages/pkg_resources/init.py", line 1501, in run_script
exec(code, namespace, namespace)
File "/pipeline/Circlator/vir_env/lib/python3.5/site-packages/circlator-1.3.1-py3.5.egg/EGG-INFO/scripts/circlator", line 58, in
exec('circlator.tasks.' + task + '.run()')
File "", line 1, in
File "/pipeline/Circlator/vir_env/lib/python3.5/site-packages/circlator-1.3.1-py3.5.egg/circlator/tasks/all.py", line 160, in run
a.run()
File "/pipeline/Circlator/vir_env/lib/python3.5/site-packages/circlator-1.3.1-py3.5.egg/circlator/assemble.py", line 123, in run
self.run_spades(stop_at_first_success=self.spades_use_first_success)
File "/pipeline/Circlator/vir_env/lib/python3.5/site-packages/circlator-1.3.1-py3.5.egg/circlator/assemble.py", line 118, in run_spades
raise Error('Error running SPAdes. Output directories are:\n ' + '\n '.join(kmer_to_dir.values()) + '\nThe reason why should be in the spades.log file in each directory.')
circlator.assemble.Error: Error running SPAdes.

Thank you for your reply!

Error when running circlator: can't find canu version

I am using the new version of canu (version 1.6).

On running circlator (version 1.5.1), I get:

circlator.external_progs.Error: Found canu but couldn't get version.
Cannot continue

Thanks.

Temporary folders not cleaned up when process killed

Does the temp directory class in Python have a way to ensure cleanup in the case of killed job (eg. CTRL-C). Can you trap the signal?

eg. pilon.fixstart.tmp.circlator.fixstart.xn4xwx51/

fixstart with user supplied gene sequences does not work

Hello,

I tried the following command to let circlator (v.1.5.1) only reset the start point of the contig using a pre-defined gene sequence but it seems that circlator just ignore the sequence that I provided and went ahead to reset start point based on Prodigal's output.

my command:
circlator fixstart --genes_fa start_gene.cds.fa --min_id 90 test.mt_contig.fa test.mt_contig.circlator

The detailed log file goes like this:
No suitable promer matches found
Using the following prodigal predictions to circularize contigs:
chr_chrMT Prodigal_v2.6.2 CDS 25812 25955 23.6 - 0 ;gc_cont=0.417;tscore=2.15;

I didn't encounter this problem with earlier versions of circlator though.

Best,
Jia-Xing

disabling mismatch correction in SPAdes

I'm running circlator with SPAdes 3.6.0 and sometimes it crashes with the following error during the contig mismatch correction at the end:

Line 6191, sequence length 17974 vs 1590 from CIGAR
Parse error at line 6191: CIGAR and sequence length are inconsistent


== Error ==  system call for: "['/home/bastian/bin/SPAdes-3.5.0-Linux/bin/corrector', '/share/project/circlator_results/03.assemble.tmp.spades.87.7bjtc3gr/mismatch_corrector/contigs/configs/corrector.info', '/share/project/circlator_results/03.assemble.tmp.spades.87.7bjtc3gr/misc/assembled_contigs.fasta']" finished abnormally, err code: -6

This leads to the whole circlator all crashing in the end, as it happens for each k-mer size. I guess I could just hard-code the corresponding flags into circlator, but I wondered whether disabling this would

a) be a good idea at all or whether it will lead to my results going down the drain
b) if this should be added as a general option to circlator if it's a good idea?

too many values to unpack error on merge command

I'm getting the following error when running circlator merge:

Traceback (most recent call last):
  File "/usr/local/bin/circlator", line 56, in <module>
    exec('circlator.tasks.' + task + '.run()')
  File "<string>", line 1, in <module>
  File "/opt/anaconda/envs/py3.3/lib/python3.3/site-packages/circlator/tasks/merge.py", line 46, in run
    m.run()
  File "/opt/anaconda/envs/py3.3/lib/python3.3/site-packages/circlator/merge.py", line 793, in run
    self.original_fasta, nucmer_coords_file, act_script = self._iterative_bridged_contig_pair_merge(self.outprefix + '.merge')
ValueError: too many values to unpack (expected 3)

The commands before merge appear to be working fine. Not sure what's going on. Any advice?

How to use Canu in Circlator all?

I keep on getting an error (unrecognized arguments: --assembler) when I use
circlator all --assembler canu --merge_min_id 85 --merge_breaklen 1000 contigs.fasta correctedReads.fasta.gz /mnt/output

Canu is in my path.

Any ideas?

version of SPAdes ?

Why do you consider SPAdes 3.7.1 to produce the optimal results for Circlator?

We have tried it with the newest SPAdes 3.10.0 with the following warning:

WARNING: SPAdes version 3.10.0 is being used. It will work, but better results are usually obtained from Circlator using SPAdes version 3.7.1. Although 3.7.1 is not the latest version, we recommend it for Circlator.

Thanks!

fixstart fails if spaces in sequence names

Also check other individual tasks. Running "circlator all" is ok because the first thing it does is fix the names in the input assembly.

circlator progcheck report PATH of binary?

Martin - could you modify circlator to print out which binary it found?
eg.

% circlator  progcheck
Using bwa version 0.7.12 as /bio/sw/bwa/bin/bwa
Using samtools version 1.2 as /usr/bin/samtools

minimus2 using hard-coded /usr/local/bin/show-coords

Command: /usr/local/bin/show-coords -H -c -l -o -r -I 94 /home/tseemann/tasks/glen.carter/canu_bad_pacbio/A01/pilon/tig00000000_pilon.seq.minimus2.merge/minimus2.delta | /bio/linuxbrew/Cellar/amos/3.1.0/bin/nucmerAnnotate | egrep 'BEGIN|END|CONTAIN|IDENTITY' > /home/tseemann/tasks/glen.carter/canu_bad_pacbio/A01/pilon/tig00000000_pilon.seq.minimus2.merge/minimus2.coords  exited with status: 1

... end of Minimus2 output.

% circlator progcheck
Using bwa version 0.7.13 as /bio/linuxbrew/bin/bwa
Using nucmer version 3.1 as /bio/linuxbrew/bin/nucmer
Using prodigal version 2.6.2 as /bio/linuxbrew/bin/prodigal
Using samtools version 1.3 as /bio/linuxbrew/bin/samtools
Using spades version .8.0 as /bio/linuxbrew/bin/spades.py

installing circlator

I think this is a trivial point that I am not being able to overcome.

I am trying to install circlator. However, didn't succeed. I downloaded the release and tried to install like this, it says python3: command not found
Suresh-Panthees-MacBook-Pro:~ Suresh$ cd desktop
Suresh-Panthees-MacBook-Pro:desktop Suresh$ cd circlator-1.5.1
Suresh-Panthees-MacBook-Pro:circlator-1.5.1 Suresh$ python3 setup.py test
-bash: python3: command not found
Suresh-Panthees-MacBook-Pro:circlator-1.5.1 Suresh$

Help please

Thanks

Suresh

Circularization of BAC contigs

To whom it may concern,

I am attempting to use the circlator for circularization of BAC contigs that has been sequenced with P6C4 chemistry and assembled with CANU. The documentation suggests that corrected reads should be used as input.

We have pooled the BAC for sequencing. If we sequenced the BAC at an average depth of 100X and if there are 100 BAC clones, the coverage for the vector sequence is 10000X. The vector sequence (7507 bp) is present in all BAC clones and vector sequence hindered the assembly process and reduced the assembly statistics. Therefore, we masked the sequences aligning to the vector sequence and as a result the reads with vector sequences have not been corrected with CANU.

I wanted to ask whether the raw subreads that aligns to the vector sequences can be effectively used for the circlator.

I look forward to your response.

Best,
Jin

Circlator using a gene evicted from Uniprot 1 year ago

This came up on one of our circlator fixstart runs on a K.pneumo chromosome:

tr|U1EGA9|U1EGA9_ENTGA

http://www.uniprot.org/uniprot/U1EGA9

This entry is obsolete
On September 16, 2015 this entry was deleted.
For previous versions of this entry, please look at its history.

(Found by @AnnaSyme)

circlator installation problem with regard to the liblzma library

Hello, I encountered the following error (see ### error log ###) when trying to install circlator on a specific server via "py3_virtualenv_circlator/bin/pip3 install "circlator==1.5.1". The installation is fine on other servers though. It appears to me the problem is the library liblzma cannot be found. But in fact this library has been installed in the system:

gilles@big-daddy:~$ /sbin/ldconfig -p|egrep liblzma
liblzma.so.5 (libc6,x86-64) => /lib/x86_64-linux-gnu/liblzma.so.5
liblzma.so.5 (libc6) => /lib/i386-linux-gnu/liblzma.so.5

Could you help to to diagnose what might be the true cause of this problem? Thx!

################# error log #############################
Downloading/unpacking circlator==1.5.1
Downloading circlator-1.5.1.tar.gz (5.3MB): 5.3MB downloaded
Running setup.py (path:/home/gilles/LRSDAY_v20170710/build/py3_virtualenv_circlator/build/circlator/setup.py) egg_info for package circlator

Downloading/unpacking openpyxl (from circlator==1.5.1)
Downloading openpyxl-2.4.8.tar.gz (156kB): 156kB downloaded
Running setup.py (path:/home/gilles/LRSDAY_v20170710/build/py3_virtualenv_circlator/build/openpyxl/setup.py) egg_info for package openpyxl

no previously-included directories found matching 'openpyxl/tests'
no previously-included directories found matching 'openpyxl/sample'
no previously-included directories found matching 'openpyxl/benchmarks'
no previously-included directories found matching 'openpyxl/develop'
no previously-included directories found matching 'scratchpad'
warning: no previously-included files matching 'test_*.py' found under directory 'openpyxl'
warning: no previously-included files matching 'tests/*.py' found under directory 'openpyxl'

Downloading/unpacking pyfastaq>=3.12.1 (from circlator==1.5.1)
Downloading pyfastaq-3.15.0.tar.gz (49kB): 49kB downloaded
Running setup.py (path:/home/gilles/LRSDAY_v20170710/build/py3_virtualenv_circlator/build/pyfastaq/setup.py) egg_info for package pyfastaq

Downloading/unpacking pysam>=0.8.1 (from circlator==1.5.1)
Downloading pysam-0.11.2.2.tar.gz (2.5MB): 2.5MB downloaded
Running setup.py (path:/home/gilles/LRSDAY_v20170710/build/py3_virtualenv_circlator/build/pysam/setup.py) egg_info for package pysam
checking for gcc... gcc
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables...
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether gcc accepts -g... yes
checking for gcc option to accept ISO C89... none needed
checking for ranlib... ranlib
checking for special C compiler options needed for large files... no
checking for _FILE_OFFSET_BITS value needed for large files... no
checking shared library type... plain .so
checking how to run the C preprocessor... gcc -E
checking for grep that handles long lines and -e... /bin/grep
checking for egrep... /bin/grep -E
checking for ANSI C header files... yes
checking for sys/types.h... yes
checking for sys/stat.h... yes
checking for stdlib.h... yes
checking for string.h... yes
checking for memory.h... yes
checking for strings.h... yes
checking for inttypes.h... yes
checking for stdint.h... yes
checking for unistd.h... yes
checking for stdlib.h... (cached) yes
checking for unistd.h... (cached) yes
checking for sys/param.h... yes
checking for getpagesize... yes
checking for working mmap... yes
checking for gmtime_r... yes
checking whether fdatasync is declared... yes
checking for fdatasync... yes
checking for library containing log... -lm
checking for zlib.h... yes
checking for inflate in -lz... yes
checking for library containing connect... none required
checking for bzlib.h... yes
checking for BZ2_bzBuffToBuffCompress in -lbz2... yes
checking for lzma.h... no
checking for lzma_easy_buffer_encode in -llzma... no
configure: error: liblzma development files not found

The CRAM format may use LZMA2 compression, which is implemented in HTSlib
by using compression routines from liblzma <http://tukaani.org/xz/>.


Building HTSlib requires liblzma development files to be installed on the
build machine; you may need to ensure a package such as liblzma-dev (on Debian
or Ubuntu Linux), xz-devel (on RPM-based Linux distributions or Cygwin), or
xz (via Homebrew on macOS) is installed; or build XZ Utils from source.


Either configure with --disable-lzma (which will make some CRAM files
produced elsewhere unreadable) or resolve this error to build HTSlib.
checking for gcc... gcc
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables...
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether gcc accepts -g... yes
checking for gcc option to accept ISO C89... none needed
checking for ranlib... ranlib
checking for special C compiler options needed for large files... no
checking for _FILE_OFFSET_BITS value needed for large files... no
checking shared library type... plain .so
checking how to run the C preprocessor... gcc -E
checking for grep that handles long lines and -e... /bin/grep
checking for egrep... /bin/grep -E
checking for ANSI C header files... yes
checking for sys/types.h... yes
checking for sys/stat.h... yes
checking for stdlib.h... yes
checking for string.h... yes
checking for memory.h... yes
checking for strings.h... yes
checking for inttypes.h... yes
checking for stdint.h... yes
checking for unistd.h... yes
checking for stdlib.h... (cached) yes
checking for unistd.h... (cached) yes
checking for sys/param.h... yes
checking for getpagesize... yes
checking for working mmap... yes
checking for gmtime_r... yes
checking whether fdatasync is declared... yes
checking for fdatasync... yes
checking for library containing log... -lm
checking for zlib.h... yes
checking for inflate in -lz... yes
checking for library containing connect... none required
checking for bzlib.h... yes
checking for BZ2_bzBuffToBuffCompress in -lbz2... yes
checking for lzma.h... no
checking for lzma_easy_buffer_encode in -llzma... no
configure: error: liblzma development files not found


The CRAM format may use LZMA2 compression, which is implemented in HTSlib
by using compression routines from liblzma <http://tukaani.org/xz/>.


Building HTSlib requires liblzma development files to be installed on the
build machine; you may need to ensure a package such as liblzma-dev (on Debian
or Ubuntu Linux), xz-devel (on RPM-based Linux distributions or Cygwin), or
xz (via Homebrew on macOS) is installed; or build XZ Utils from source.


Either configure with --disable-lzma (which will make some CRAM files
produced elsewhere unreadable) or resolve this error to build HTSlib.
# pysam: no cython available - using pre-compiled C
# pysam: htslib mode is shared
# pysam: HTSLIB_CONFIGURE_OPTIONS=None
# pysam: htslib configure options: None
# pysam: htslib_config PLATFORM=Linux
# pysam: htslib_config LIBHTS_OBJS=kfunc.o knetfile.o kstring.o bcf_sr_sort.o bgzf.o errmod.o faidx.o hfile.o hfile_net.o hts.o md5.o multipart.o probaln.o realn.o regidx.o sam.o synced_bcf_reader.o vcf_sweep.o tbx.o textutils.o thread_pool.o vcf.o vcfutils.o cram/cram_codecs.o cram/cram_decode.o cram/cram_encode.o cram/cram_external.o cram/cram_index.o cram/cram_io.o cram/cram_samtools.o cram/cram_stats.o cram/files.o cram/mFILE.o cram/open_trace_file.o cram/pooled_alloc.o cram/rANS_static.o cram/sam_header.o cram/string_alloc.o cram/vlen.o cram/zfio.o
# pysam: htslib_config LDFLAGS=
# pysam: htslib_config LIBS=-lz -lm -lbz2 -llzma
# pysam: config_option: ENABLE_PLUGINS=0
# pysam: config_option: HAVE_COMMONCRYPTO=0
# pysam: config_option: HAVE_GMTIME_R=0
# pysam: config_option: HAVE_HMAC=0
# pysam: config_option: HAVE_IRODS=0
# pysam: config_option: HAVE_LIBCURL=0
# pysam: config_option: HAVE_MMAP=0


package init file 'samtools/__init__.py' not found (or not a regular file)
package init file 'bcftools/__init__.py' not found (or not a regular file)
package init file 'samtools/win32/__init__.py' not found (or not a regular file)
package init file 'htslib/__init__.py' not found (or not a regular file)
package init file 'htslib/htslib/__init__.py' not found (or not a regular file)
warning: no files found matching 'KNOWN_BUGS'
warning: no files found matching 'THANKS'
warning: no files found matching 'samtools/configure'
warning: no files found matching 'samtools/config.mk.in'
warning: no files found matching 'samtools/config.h.in'
warning: no files found matching 'tests/pysam_data'
warning: no files found matching 'tests/tabix_data'

Downloading/unpacking pymummer>=0.9.0 (from circlator==1.5.1)
Downloading pymummer-0.10.2.tar.gz
Running setup.py (path:/home/gilles/LRSDAY_v20170710/build/py3_virtualenv_circlator/build/pymummer/setup.py) egg_info for package pymummer
Checking MUMmer programs found in path:
OK nucmer
OK show-coords
OK show-snps
OK delta-filter

Downloading/unpacking jdcal (from openpyxl->circlator==1.5.1)
Downloading jdcal-1.3.tar.gz
Running setup.py (path:/home/gilles/LRSDAY_v20170710/build/py3_virtualenv_circlator/build/jdcal/setup.py) egg_info for package jdcal

Downloading/unpacking et-xmlfile (from openpyxl->circlator==1.5.1)
Downloading et_xmlfile-1.0.1.tar.gz
Running setup.py (path:/home/gilles/LRSDAY_v20170710/build/py3_virtualenv_circlator/build/et-xmlfile/setup.py) egg_info for package et-xmlfile

Installing collected packages: circlator, openpyxl, pyfastaq, pysam, pymummer, jdcal, et-xmlfile
Running setup.py install for circlator
changing mode of build/scripts-3.4/circlator from 664 to 775

changing mode of /home/gilles/LRSDAY_v20170710/build/py3_virtualenv_circlator/bin/circlator to 775

Running setup.py install for openpyxl

no previously-included directories found matching 'openpyxl/tests'
no previously-included directories found matching 'openpyxl/sample'
no previously-included directories found matching 'openpyxl/benchmarks'
no previously-included directories found matching 'openpyxl/develop'
no previously-included directories found matching 'scratchpad'
warning: no previously-included files matching 'test_*.py' found under directory 'openpyxl'
warning: no previously-included files matching 'tests/*.py' found under directory 'openpyxl'

Running setup.py install for pyfastaq
changing mode of build/scripts-3.4/fastaq from 664 to 775

changing mode of /home/gilles/LRSDAY_v20170710/build/py3_virtualenv_circlator/bin/fastaq to 775

Running setup.py install for pysam
checking for gcc... gcc
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables...
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether gcc accepts -g... yes
checking for gcc option to accept ISO C89... none needed
checking for ranlib... ranlib
checking for special C compiler options needed for large files... no
checking for _FILE_OFFSET_BITS value needed for large files... no
checking shared library type... plain .so
checking how to run the C preprocessor... gcc -E
checking for grep that handles long lines and -e... /bin/grep
checking for egrep... /bin/grep -E
checking for ANSI C header files... yes
checking for sys/types.h... yes
checking for sys/stat.h... yes
checking for stdlib.h... yes
checking for string.h... yes
checking for memory.h... yes
checking for strings.h... yes
checking for inttypes.h... yes
checking for stdint.h... yes
checking for unistd.h... yes
checking for stdlib.h... (cached) yes
checking for unistd.h... (cached) yes
checking for sys/param.h... yes
checking for getpagesize... yes
checking for working mmap... yes
checking for gmtime_r... yes
checking whether fdatasync is declared... yes
checking for fdatasync... yes
checking for library containing log... -lm
checking for zlib.h... yes
checking for inflate in -lz... yes
checking for library containing connect... none required
checking for bzlib.h... yes
checking for BZ2_bzBuffToBuffCompress in -lbz2... yes
checking for lzma.h... no
checking for lzma_easy_buffer_encode in -llzma... no
configure: error: liblzma development files not found

The CRAM format may use LZMA2 compression, which is implemented in HTSlib
by using compression routines from liblzma <http://tukaani.org/xz/>.


Building HTSlib requires liblzma development files to be installed on the
build machine; you may need to ensure a package such as liblzma-dev (on Debian
or Ubuntu Linux), xz-devel (on RPM-based Linux distributions or Cygwin), or
xz (via Homebrew on macOS) is installed; or build XZ Utils from source.


Either configure with --disable-lzma (which will make some CRAM files
produced elsewhere unreadable) or resolve this error to build HTSlib.
checking for gcc... gcc
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables...
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether gcc accepts -g... yes
checking for gcc option to accept ISO C89... none needed
checking for ranlib... ranlib
checking for special C compiler options needed for large files... no
checking for _FILE_OFFSET_BITS value needed for large files... no
checking shared library type... plain .so
checking how to run the C preprocessor... gcc -E
checking for grep that handles long lines and -e... /bin/grep
checking for egrep... /bin/grep -E
checking for ANSI C header files... yes
checking for sys/types.h... yes
checking for sys/stat.h... yes
checking for stdlib.h... yes
checking for string.h... yes
checking for memory.h... yes
checking for strings.h... yes
checking for inttypes.h... yes
checking for stdint.h... yes
checking for unistd.h... yes
checking for stdlib.h... (cached) yes
checking for unistd.h... (cached) yes
checking for sys/param.h... yes
checking for getpagesize... yes
checking for working mmap... yes
checking for gmtime_r... yes
checking whether fdatasync is declared... yes
checking for fdatasync... yes
checking for library containing log... -lm
checking for zlib.h... yes
checking for inflate in -lz... yes
checking for library containing connect... none required
checking for bzlib.h... yes
checking for BZ2_bzBuffToBuffCompress in -lbz2... yes
checking for lzma.h... no
checking for lzma_easy_buffer_encode in -llzma... no
configure: error: liblzma development files not found


The CRAM format may use LZMA2 compression, which is implemented in HTSlib
by using compression routines from liblzma <http://tukaani.org/xz/>.


Building HTSlib requires liblzma development files to be installed on the
build machine; you may need to ensure a package such as liblzma-dev (on Debian
or Ubuntu Linux), xz-devel (on RPM-based Linux distributions or Cygwin), or
xz (via Homebrew on macOS) is installed; or build XZ Utils from source.


Either configure with --disable-lzma (which will make some CRAM files
produced elsewhere unreadable) or resolve this error to build HTSlib.
# pysam: no cython available - using pre-compiled C
# pysam: htslib mode is shared
# pysam: HTSLIB_CONFIGURE_OPTIONS=None
# pysam: htslib configure options: None
# pysam: htslib_config LIBS=-lz -lm -lbz2 -llzma
# pysam: htslib_config LIBHTS_OBJS=kfunc.o knetfile.o kstring.o bcf_sr_sort.o bgzf.o errmod.o faidx.o hfile.o hfile_net.o hts.o md5.o multipart.o probaln.o realn.o regidx.o sam.o synced_bcf_reader.o vcf_sweep.o tbx.o textutils.o thread_pool.o vcf.o vcfutils.o cram/cram_codecs.o cram/cram_decode.o cram/cram_encode.o cram/cram_external.o cram/cram_index.o cram/cram_io.o cram/cram_samtools.o cram/cram_stats.o cram/files.o cram/mFILE.o cram/open_trace_file.o cram/pooled_alloc.o cram/rANS_static.o cram/sam_header.o cram/string_alloc.o cram/vlen.o cram/zfio.o
# pysam: htslib_config PLATFORM=Linux
# pysam: htslib_config LDFLAGS=
# pysam: config_option: ENABLE_PLUGINS=0
# pysam: config_option: HAVE_COMMONCRYPTO=0
# pysam: config_option: HAVE_GMTIME_R=0
# pysam: config_option: HAVE_HMAC=0
# pysam: config_option: HAVE_IRODS=0
# pysam: config_option: HAVE_LIBCURL=0
# pysam: config_option: HAVE_MMAP=0
package init file 'samtools/__init__.py' not found (or not a regular file)
package init file 'bcftools/__init__.py' not found (or not a regular file)
package init file 'samtools/win32/__init__.py' not found (or not a regular file)
package init file 'htslib/__init__.py' not found (or not a regular file)
package init file 'htslib/htslib/__init__.py' not found (or not a regular file)
Fixing build/lib.linux-x86_64-3.4/pysam/config.py build/lib.linux-x86_64-3.4/pysam/bcftools.py build/lib.linux-x86_64-3.4/pysam/utils.py build/lib.linux-x86_64-3.4/pysam/__init__.py build/lib.linux-x86_64-3.4/pysam/version.py build/lib.linux-x86_64-3.4/pysam/Pileup.py build/lib.linux-x86_64-3.4/pysam/samtools.py build/lib.linux-x86_64-3.4/pysam/namedtuple.py build/lib.linux-x86_64-3.4/pysam/include/__init__.py
Skipping implicit fixer: buffer
Skipping implicit fixer: idioms
Skipping implicit fixer: set_literal
Skipping implicit fixer: ws_comma
Fixing build/lib.linux-x86_64-3.4/pysam/config.py build/lib.linux-x86_64-3.4/pysam/bcftools.py build/lib.linux-x86_64-3.4/pysam/utils.py build/lib.linux-x86_64-3.4/pysam/__init__.py build/lib.linux-x86_64-3.4/pysam/version.py build/lib.linux-x86_64-3.4/pysam/Pileup.py build/lib.linux-x86_64-3.4/pysam/samtools.py build/lib.linux-x86_64-3.4/pysam/namedtuple.py build/lib.linux-x86_64-3.4/pysam/include/__init__.py
Skipping implicit fixer: buffer
Skipping implicit fixer: idioms
Skipping implicit fixer: set_literal
Skipping implicit fixer: ws_comma
building 'pysam.libchtslib' extension
x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -g -fstack-protector --param=ssp-buffer-size=4 -Wformat -Werror=format-security -D_FORTIFY_SOURCE=2 -fPIC -Ipysam -I. -Ihtslib -I/usr/include/python3.4m -I/home/gilles/LRSDAY_v20170710/build/py3_virtualenv_circlator/include/python3.4m -c pysam/libchtslib.c -o build/temp.linux-x86_64-3.4/pysam/libchtslib.o -Wno-unused -Wno-strict-prototypes -Wno-sign-compare -Wno-error=declaration-after-statement
pysam/libchtslib.c:4:20: fatal error: Python.h: No such file or directory
compilation terminated.
error: command 'x86_64-linux-gnu-gcc' failed with exit status 1
Complete output from command /home/gilles/LRSDAY_v20170710/build/py3_virtualenv_circlator/bin/python3 -c "import setuptools, tokenize;__file__='/home/gilles/LRSDAY_v20170710/build/py3_virtualenv_circlator/build/pysam/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-diwk06nb-record/install-record.txt --single-version-externally-managed --compile --install-headers /home/gilles/LRSDAY_v20170710/build/py3_virtualenv_circlator/include/site/python3.4:
checking for gcc... gcc

checking whether the C compiler works... yes

checking for C compiler default output file name... a.out

checking for suffix of executables...

checking whether we are cross compiling... no

checking for suffix of object files... o

checking whether we are using the GNU C compiler... yes

checking whether gcc accepts -g... yes

checking for gcc option to accept ISO C89... none needed

checking for ranlib... ranlib

checking for special C compiler options needed for large files... no

checking for _FILE_OFFSET_BITS value needed for large files... no

checking shared library type... plain .so

checking how to run the C preprocessor... gcc -E

checking for grep that handles long lines and -e... /bin/grep

checking for egrep... /bin/grep -E

checking for ANSI C header files... yes

checking for sys/types.h... yes

checking for sys/stat.h... yes

checking for stdlib.h... yes

checking for string.h... yes

checking for memory.h... yes

checking for strings.h... yes

checking for inttypes.h... yes

checking for stdint.h... yes

checking for unistd.h... yes

checking for stdlib.h... (cached) yes

checking for unistd.h... (cached) yes

checking for sys/param.h... yes

checking for getpagesize... yes

checking for working mmap... yes

checking for gmtime_r... yes

checking whether fdatasync is declared... yes

checking for fdatasync... yes

checking for library containing log... -lm

checking for zlib.h... yes

checking for inflate in -lz... yes

checking for library containing connect... none required

checking for bzlib.h... yes

checking for BZ2_bzBuffToBuffCompress in -lbz2... yes

checking for lzma.h... no

checking for lzma_easy_buffer_encode in -llzma... no

configure: error: liblzma development files not found

The CRAM format may use LZMA2 compression, which is implemented in HTSlib

by using compression routines from liblzma http://tukaani.org/xz/.

Building HTSlib requires liblzma development files to be installed on the

build machine; you may need to ensure a package such as liblzma-dev (on Debian

or Ubuntu Linux), xz-devel (on RPM-based Linux distributions or Cygwin), or

xz (via Homebrew on macOS) is installed; or build XZ Utils from source.

Either configure with --disable-lzma (which will make some CRAM files

produced elsewhere unreadable) or resolve this error to build HTSlib.

checking for gcc... gcc

checking whether the C compiler works... yes

checking for C compiler default output file name... a.out

checking for suffix of executables...

checking whether we are cross compiling... no

checking for suffix of object files... o

checking whether we are using the GNU C compiler... yes

checking whether gcc accepts -g... yes

checking for gcc option to accept ISO C89... none needed

checking for ranlib... ranlib

checking for special C compiler options needed for large files... no

checking for _FILE_OFFSET_BITS value needed for large files... no

checking shared library type... plain .so

checking how to run the C preprocessor... gcc -E

checking for grep that handles long lines and -e... /bin/grep

checking for egrep... /bin/grep -E

checking for ANSI C header files... yes

checking for sys/types.h... yes

checking for sys/stat.h... yes

checking for stdlib.h... yes

checking for string.h... yes

checking for memory.h... yes

checking for strings.h... yes

checking for inttypes.h... yes

checking for stdint.h... yes

checking for unistd.h... yes

checking for stdlib.h... (cached) yes

checking for unistd.h... (cached) yes

checking for sys/param.h... yes

checking for getpagesize... yes

checking for working mmap... yes

checking for gmtime_r... yes

checking whether fdatasync is declared... yes

checking for fdatasync... yes

checking for library containing log... -lm

checking for zlib.h... yes

checking for inflate in -lz... yes

checking for library containing connect... none required

checking for bzlib.h... yes

checking for BZ2_bzBuffToBuffCompress in -lbz2... yes

checking for lzma.h... no

checking for lzma_easy_buffer_encode in -llzma... no

configure: error: liblzma development files not found

The CRAM format may use LZMA2 compression, which is implemented in HTSlib

by using compression routines from liblzma http://tukaani.org/xz/.

Building HTSlib requires liblzma development files to be installed on the

build machine; you may need to ensure a package such as liblzma-dev (on Debian

or Ubuntu Linux), xz-devel (on RPM-based Linux distributions or Cygwin), or

xz (via Homebrew on macOS) is installed; or build XZ Utils from source.

Either configure with --disable-lzma (which will make some CRAM files

produced elsewhere unreadable) or resolve this error to build HTSlib.

pysam: no cython available - using pre-compiled C

pysam: htslib mode is shared

pysam: HTSLIB_CONFIGURE_OPTIONS=None

pysam: htslib configure options: None

pysam: htslib_config LIBS=-lz -lm -lbz2 -llzma

pysam: htslib_config LIBHTS_OBJS=kfunc.o knetfile.o kstring.o bcf_sr_sort.o bgzf.o errmod.o faidx.o hfile.o hfile_net.o hts.o md5.o multipart.o probaln.o realn.o regidx.o sam.o synced_bcf_reader.o vcf_sweep.o tbx.o textutils.o thread_pool.o vcf.o vcfutils.o cram/cram_codecs.o cram/cram_decode.o cram/cram_encode.o cram/cram_external.o cram/cram_index.o cram/cram_io.o cram/cram_samtools.o cram/cram_stats.o cram/files.o cram/mFILE.o cram/open_trace_file.o cram/pooled_alloc.o cram/rANS_static.o cram/sam_header.o cram/string_alloc.o cram/vlen.o cram/zfio.o

pysam: htslib_config PLATFORM=Linux

pysam: htslib_config LDFLAGS=

pysam: config_option: ENABLE_PLUGINS=0

pysam: config_option: HAVE_COMMONCRYPTO=0

pysam: config_option: HAVE_GMTIME_R=0

pysam: config_option: HAVE_HMAC=0

pysam: config_option: HAVE_IRODS=0

pysam: config_option: HAVE_LIBCURL=0

pysam: config_option: HAVE_MMAP=0

running install

running build

running build_py

creating build

creating build/lib.linux-x86_64-3.4

creating build/lib.linux-x86_64-3.4/pysam

copying pysam/config.py -> build/lib.linux-x86_64-3.4/pysam

copying pysam/bcftools.py -> build/lib.linux-x86_64-3.4/pysam

copying pysam/utils.py -> build/lib.linux-x86_64-3.4/pysam

copying pysam/init.py -> build/lib.linux-x86_64-3.4/pysam

copying pysam/version.py -> build/lib.linux-x86_64-3.4/pysam

copying pysam/Pileup.py -> build/lib.linux-x86_64-3.4/pysam

copying pysam/samtools.py -> build/lib.linux-x86_64-3.4/pysam

copying pysam/namedtuple.py -> build/lib.linux-x86_64-3.4/pysam

creating build/lib.linux-x86_64-3.4/pysam/include

copying pysam/include/init.py -> build/lib.linux-x86_64-3.4/pysam/include

package init file 'samtools/init.py' not found (or not a regular file)

package init file 'bcftools/init.py' not found (or not a regular file)

package init file 'samtools/win32/init.py' not found (or not a regular file)

package init file 'htslib/init.py' not found (or not a regular file)

package init file 'htslib/htslib/init.py' not found (or not a regular file)

copying pysam/libcbcf.pxd -> build/lib.linux-x86_64-3.4/pysam

copying pysam/libcfaidx.pxd -> build/lib.linux-x86_64-3.4/pysam

copying pysam/libcsamtools.pxd -> build/lib.linux-x86_64-3.4/pysam

copying pysam/libchtslib.pxd -> build/lib.linux-x86_64-3.4/pysam

copying pysam/libctabix.pxd -> build/lib.linux-x86_64-3.4/pysam

copying pysam/libctabixproxies.pxd -> build/lib.linux-x86_64-3.4/pysam

copying pysam/libcalignedsegment.pxd -> build/lib.linux-x86_64-3.4/pysam

copying pysam/libcvcf.pxd -> build/lib.linux-x86_64-3.4/pysam

copying pysam/libcbcftools.pxd -> build/lib.linux-x86_64-3.4/pysam

copying pysam/libcalignmentfile.pxd -> build/lib.linux-x86_64-3.4/pysam

copying pysam/libcutils.pxd -> build/lib.linux-x86_64-3.4/pysam

copying pysam/libcsamfile.pxd -> build/lib.linux-x86_64-3.4/pysam

copying pysam/samfile_util.h -> build/lib.linux-x86_64-3.4/pysam

copying pysam/tabix_util.h -> build/lib.linux-x86_64-3.4/pysam

copying pysam/cbcftools_util.h -> build/lib.linux-x86_64-3.4/pysam

copying pysam/csamtools_util.h -> build/lib.linux-x86_64-3.4/pysam

copying pysam/pysam_util.h -> build/lib.linux-x86_64-3.4/pysam

copying pysam/htslib_util.h -> build/lib.linux-x86_64-3.4/pysam

copying pysam/pysam_stream.h -> build/lib.linux-x86_64-3.4/pysam

creating build/lib.linux-x86_64-3.4/pysam/include/samtools

copying samtools/bam2bcf.h -> build/lib.linux-x86_64-3.4/pysam/include/samtools

copying samtools/bam_endian.h -> build/lib.linux-x86_64-3.4/pysam/include/samtools

copying samtools/bam_plbuf.h -> build/lib.linux-x86_64-3.4/pysam/include/samtools

copying samtools/config.h -> build/lib.linux-x86_64-3.4/pysam/include/samtools

copying samtools/bam.h -> build/lib.linux-x86_64-3.4/pysam/include/samtools

copying samtools/sample.h -> build/lib.linux-x86_64-3.4/pysam/include/samtools

copying samtools/version.h -> build/lib.linux-x86_64-3.4/pysam/include/samtools

copying samtools/samtools.h -> build/lib.linux-x86_64-3.4/pysam/include/samtools

copying samtools/sam_header.h -> build/lib.linux-x86_64-3.4/pysam/include/samtools

copying samtools/sam_opts.h -> build/lib.linux-x86_64-3.4/pysam/include/samtools

copying samtools/pysam.h -> build/lib.linux-x86_64-3.4/pysam/include/samtools

copying samtools/bam_lpileup.h -> build/lib.linux-x86_64-3.4/pysam/include/samtools

copying samtools/sam.h -> build/lib.linux-x86_64-3.4/pysam/include/samtools

copying samtools/stats_isize.h -> build/lib.linux-x86_64-3.4/pysam/include/samtools

creating build/lib.linux-x86_64-3.4/pysam/include/bcftools

copying bcftools/khash_str2str.h -> build/lib.linux-x86_64-3.4/pysam/include/bcftools

copying bcftools/call.h -> build/lib.linux-x86_64-3.4/pysam/include/bcftools

copying bcftools/bam2bcf.h -> build/lib.linux-x86_64-3.4/pysam/include/bcftools

copying bcftools/convert.h -> build/lib.linux-x86_64-3.4/pysam/include/bcftools

copying bcftools/vcmp.h -> build/lib.linux-x86_64-3.4/pysam/include/bcftools

copying bcftools/rbuf.h -> build/lib.linux-x86_64-3.4/pysam/include/bcftools

copying bcftools/ploidy.h -> build/lib.linux-x86_64-3.4/pysam/include/bcftools

copying bcftools/kmin.h -> build/lib.linux-x86_64-3.4/pysam/include/bcftools

copying bcftools/HMM.h -> build/lib.linux-x86_64-3.4/pysam/include/bcftools

copying bcftools/mw.h -> build/lib.linux-x86_64-3.4/pysam/include/bcftools

copying bcftools/version.h -> build/lib.linux-x86_64-3.4/pysam/include/bcftools

copying bcftools/filter.h -> build/lib.linux-x86_64-3.4/pysam/include/bcftools

copying bcftools/tsv2vcf.h -> build/lib.linux-x86_64-3.4/pysam/include/bcftools

copying bcftools/gvcf.h -> build/lib.linux-x86_64-3.4/pysam/include/bcftools

copying bcftools/hclust.h -> build/lib.linux-x86_64-3.4/pysam/include/bcftools

copying bcftools/bam_sample.h -> build/lib.linux-x86_64-3.4/pysam/include/bcftools

copying bcftools/regidx.h -> build/lib.linux-x86_64-3.4/pysam/include/bcftools

copying bcftools/prob1.h -> build/lib.linux-x86_64-3.4/pysam/include/bcftools

copying bcftools/bcftools.h -> build/lib.linux-x86_64-3.4/pysam/include/bcftools

copying bcftools/kheap.h -> build/lib.linux-x86_64-3.4/pysam/include/bcftools

copying bcftools/pysam.h -> build/lib.linux-x86_64-3.4/pysam/include/bcftools

copying bcftools/bin.h -> build/lib.linux-x86_64-3.4/pysam/include/bcftools

copying bcftools/smpl_ilist.h -> build/lib.linux-x86_64-3.4/pysam/include/bcftools

creating build/lib.linux-x86_64-3.4/pysam/include/samtools/win32

copying samtools/win32/zconf.h -> build/lib.linux-x86_64-3.4/pysam/include/samtools/win32

copying samtools/win32/xcurses.h -> build/lib.linux-x86_64-3.4/pysam/include/samtools/win32

copying samtools/win32/zlib.h -> build/lib.linux-x86_64-3.4/pysam/include/samtools/win32

creating build/lib.linux-x86_64-3.4/pysam/include/htslib

copying htslib/config.h -> build/lib.linux-x86_64-3.4/pysam/include/htslib

copying htslib/bcf_sr_sort.h -> build/lib.linux-x86_64-3.4/pysam/include/htslib

copying htslib/version.h -> build/lib.linux-x86_64-3.4/pysam/include/htslib

copying htslib/hts_internal.h -> build/lib.linux-x86_64-3.4/pysam/include/htslib

copying htslib/thread_pool_internal.h -> build/lib.linux-x86_64-3.4/pysam/include/htslib

copying htslib/hfile_internal.h -> build/lib.linux-x86_64-3.4/pysam/include/htslib

creating build/lib.linux-x86_64-3.4/pysam/include/htslib/htslib

copying htslib/htslib/faidx.h -> build/lib.linux-x86_64-3.4/pysam/include/htslib/htslib

copying htslib/htslib/kstring.h -> build/lib.linux-x86_64-3.4/pysam/include/htslib/htslib

copying htslib/htslib/vcfutils.h -> build/lib.linux-x86_64-3.4/pysam/include/htslib/htslib

copying htslib/htslib/bgzf.h -> build/lib.linux-x86_64-3.4/pysam/include/htslib/htslib

copying htslib/htslib/synced_bcf_reader.h -> build/lib.linux-x86_64-3.4/pysam/include/htslib/htslib

copying htslib/htslib/tbx.h -> build/lib.linux-x86_64-3.4/pysam/include/htslib/htslib

copying htslib/htslib/knetfile.h -> build/lib.linux-x86_64-3.4/pysam/include/htslib/htslib

copying htslib/htslib/hfile.h -> build/lib.linux-x86_64-3.4/pysam/include/htslib/htslib

copying htslib/htslib/khash_str2int.h -> build/lib.linux-x86_64-3.4/pysam/include/htslib/htslib

copying htslib/htslib/hts_defs.h -> build/lib.linux-x86_64-3.4/pysam/include/htslib/htslib

copying htslib/htslib/thread_pool.h -> build/lib.linux-x86_64-3.4/pysam/include/htslib/htslib

copying htslib/htslib/regidx.h -> build/lib.linux-x86_64-3.4/pysam/include/htslib/htslib

copying htslib/htslib/vcf.h -> build/lib.linux-x86_64-3.4/pysam/include/htslib/htslib

copying htslib/htslib/vcf_sweep.h -> build/lib.linux-x86_64-3.4/pysam/include/htslib/htslib

copying htslib/htslib/hts_endian.h -> build/lib.linux-x86_64-3.4/pysam/include/htslib/htslib

copying htslib/htslib/ksort.h -> build/lib.linux-x86_64-3.4/pysam/include/htslib/htslib

copying htslib/htslib/kbitset.h -> build/lib.linux-x86_64-3.4/pysam/include/htslib/htslib

copying htslib/htslib/khash.h -> build/lib.linux-x86_64-3.4/pysam/include/htslib/htslib

copying htslib/htslib/sam.h -> build/lib.linux-x86_64-3.4/pysam/include/htslib/htslib

copying htslib/htslib/hts.h -> build/lib.linux-x86_64-3.4/pysam/include/htslib/htslib

copying htslib/htslib/klist.h -> build/lib.linux-x86_64-3.4/pysam/include/htslib/htslib

copying htslib/htslib/kseq.h -> build/lib.linux-x86_64-3.4/pysam/include/htslib/htslib

copying htslib/htslib/kfunc.h -> build/lib.linux-x86_64-3.4/pysam/include/htslib/htslib

copying htslib/htslib/cram.h -> build/lib.linux-x86_64-3.4/pysam/include/htslib/htslib

Fixing build/lib.linux-x86_64-3.4/pysam/config.py build/lib.linux-x86_64-3.4/pysam/bcftools.py build/lib.linux-x86_64-3.4/pysam/utils.py build/lib.linux-x86_64-3.4/pysam/init.py build/lib.linux-x86_64-3.4/pysam/version.py build/lib.linux-x86_64-3.4/pysam/Pileup.py build/lib.linux-x86_64-3.4/pysam/samtools.py build/lib.linux-x86_64-3.4/pysam/namedtuple.py build/lib.linux-x86_64-3.4/pysam/include/init.py

Skipping implicit fixer: buffer

Skipping implicit fixer: idioms

Skipping implicit fixer: set_literal

Skipping implicit fixer: ws_comma

Skipping implicit fixer: buffer

Skipping implicit fixer: idioms

Skipping implicit fixer: set_literal

Skipping implicit fixer: ws_comma

running build_ext

building 'pysam.libchtslib' extension

creating build/temp.linux-x86_64-3.4

creating build/temp.linux-x86_64-3.4/pysam

creating build/temp.linux-x86_64-3.4/htslib

creating build/temp.linux-x86_64-3.4/htslib/cram

x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -g -fstack-protector --param=ssp-buffer-size=4 -Wformat -Werror=format-security -D_FORTIFY_SOURCE=2 -fPIC -Ipysam -I. -Ihtslib -I/usr/include/python3.4m -I/home/gilles/LRSDAY_v20170710/build/py3_virtualenv_circlator/include/python3.4m -c pysam/libchtslib.c -o build/temp.linux-x86_64-3.4/pysam/libchtslib.o -Wno-unused -Wno-strict-prototypes -Wno-sign-compare -Wno-error=declaration-after-statement

pysam/libchtslib.c:4:20: fatal error: Python.h: No such file or directory

compilation terminated.

error: command 'x86_64-linux-gnu-gcc' failed with exit status 1

failure to circularise: cannot use this pair because longer match was found

Hi,

I am trying to use circlator to circularise my pacbio bacterial genomes. The genome has 1 chromosome and 5 plasmids. Using the default command "circlator all" it successfully circularises the chromosome and 4 of the contigs. However one contig (tig00000009) is failing to circularise, and I'm not sure why.

In the 04.merge.circularise_details.log file it says "cannot use this pair because longer match was found", I assume this is where it is going wrong. Could you explain what this means?

Here is the log file:

[merge circularise_details] [merge circularise_details] [merge circularise_details] [merge circularise_details] [merge circularise_details] [merge circularise_details] [merge circularise_details] [merge circularise_details] [merge circularise_details] [merge circularise_details] [merge circularise_details] [merge circularise_details] [merge circularise_details] [merge circularise_details] [merge circularise_details] [merge circularise_details] [merge circularise_details] [merge circularise_details] [merge circularise_details] [merge circularise_details] [merge circularise_details] [merge circularise_details] [merge circularise_details] [merge circularise_details] [merge circularise_details] [merge circularise_details] [merge circularise_details] [merge circularise_details] [merge circularise_details] [merge circularise_details] [merge circularise_details]
[merge circularise_details] [merge circularise_details]
[merge circularise_details] [merge circularise_details] [merge circularise_details] [merge circularise_details] [merge circularise_details] [merge circularise_details]
[merge circularise_details] [merge circularise_details] [merge circularise_details] [merge circularise_details] [merge circularise_details] [merge circularise_details] [merge circularise_details]
[merge circularise_details] [merge circularise_details] [merge circularise_details] [merge circularise_details] [merge circularise_details] [merge circularise_details] [merge circularise_details] [merge circularise_details]
[merge circularise_details] [merge circularise_details] [merge circularise_details] [merge circularise_details] [merge circularise_details] [merge circularise_details] [merge circularise_details] [merge circularise_details]
[merge circularise_details] [merge circularise_details] [merge circularise_details] [merge circularise_details] [merge circularise_details] [merge circularise_details] [merge circularise_details] [merge circularise_details]
[merge circularise_details] [merge circularise_details] [merge circularise_details] [merge circularise_details] [merge circularise_details] [merge circularise_details] [merge circularise_details] tig00000009 Checking 3 nucmer hits
tig00000009 potential pair of nucmer hits for circularization:
tig00000009 1 13475 13476 1 13475 13476 99.99 39155 21213 1 tig00000009 NODE_6_length_21213_cov_79.7433_ID_37074
tig00000009 33247 39155 21213 15305 5909 5909 100.00 39155 21213 1 tig00000009 NODE_6_length_21213_cov_79.7433_ID_37074
tig00000009 cannot use this pair because longer match was found
tig00000008 Checking 2 nucmer hits
tig00000008 potential pair of nucmer hits for circularization:
tig00000008 1 30669 12112 42783 30669 30672 99.99 62583 42783 1 tig00000008 NODE_5_length_42783_cov_88.7264_ID_37072
tig00000008 30593 62583 1 31996 31991 31996 99.97 62583 42783 1 tig00000008 NODE_5_length_42783_cov_88.7264_ID_37072
tig00000008 can use this pair of hits
tig00000000 Checking 2 nucmer hits
tig00000000 potential pair of nucmer hits for circularization:
tig00000000 1 50000 16397 66396 50000 50000 100.00 6257505 66396 1 tig00000000 NODE_3_length_66396_cov_30.9833_ID_37068
tig00000000 6226450 6257505 1 31056 31056 31056 100.00 6257505 66396 1 tig00000000 NODE_3_length_66396_cov_30.9833_ID_37068
tig00000000 can use this pair of hits
tig00000005 Checking 2 nucmer hits
tig00000005 potential pair of nucmer hits for circularization:
tig00000005 1 48690 20084 68774 48690 48691 99.99 124726 68774 1 tig00000005 NODE_2_length_68774_cov_97.0195_ID_37066
tig00000005 82779 124726 1 41948 41948 41948 100.00 124726 68774 1 tig00000005 NODE_2_length_68774_cov_97.0195_ID_37066
tig00000005 can use this pair of hits
tig00000004 Checking 3 nucmer hits
tig00000004 potential pair of nucmer hits for circularization:
tig00000004 1 49612 16420 66030 49612 49611 99.99 117048 66030 1 tig00000004 NODE_4_length_66030_cov_80.4157_ID_37070
tig00000004 81423 117048 1 35629 35626 35629 99.99 117048 66030 1 tig00000004 NODE_4_length_66030_cov_80.4157_ID_37070
tig00000004 can use this pair of hits
tig00000006 Checking 5 nucmer hits
tig00000006 potential pair of nucmer hits for circularization:
tig00000006 3332 63098 59767 1 59767 59767 100.00 91442 69519 1 tig00000006 NODE_1_length_69519_cov_84.964_ID_37064
tig00000006 61023 91442 69519 39100 30420 30420 100.00 91442 69519 1 tig00000006 NODE_1_length_69519_cov_84.964_ID_37064
tig00000006 can use this pair of hits
SPAdes reassembly contigs that are circular: None
tig00000009 Trying to circularize. Has nucmer hits to check...
tig00000009 No matches to SPAdes circular contigs
tig00000009 Could not circularize using matches to SPAdes circular contigs
tig00000009 Cannot circularize: no suitable nucmer hits
tig00000009 Circularized: no
tig00000008 Trying to circularize. Has nucmer hits to check...
tig00000008 Could not circularize using matches to SPAdes circular contigs
tig00000008 Circularizing using this pair of nucmer matches to SPAdes contig:
tig00000008 1 30669 12112 42783 30669 30672 99.99 62583 42783 1 tig00000008 NODE_5_length_42783_cov_88.7264_ID_37072
tig00000008 30593 62583 1 31996 31991 31996 99.97 62583 42783 1 tig00000008 NODE_5_length_42783_cov_88.7264_ID_37072
tig00000008 Circularized: yes
tig00000000 Trying to circularize. Has nucmer hits to check...
tig00000000 No matches to SPAdes circular contigs
tig00000000 Could not circularize using matches to SPAdes circular contigs
tig00000000 Circularizing using this pair of nucmer matches to SPAdes contig:
tig00000000 1 50000 16397 66396 50000 50000 100.00 6257505 66396 1 tig00000000 NODE_3_length_66396_cov_30.9833_ID_37068
tig00000000 6226450 6257505 1 31056 31056 31056 100.00 6257505 66396 1 tig00000000 NODE_3_length_66396_cov_30.9833_ID_37068
tig00000000 Circularized: yes
tig00000005 Trying to circularize. Has nucmer hits to check...
tig00000005 No matches to SPAdes circular contigs
tig00000005 Could not circularize using matches to SPAdes circular contigs
tig00000005 Circularizing using this pair of nucmer matches to SPAdes contig:
tig00000005 1 48690 20084 68774 48690 48691 99.99 124726 68774 1 tig00000005 NODE_2_length_68774_cov_97.0195_ID_37066
tig00000005 82779 124726 1 41948 41948 41948 100.00 124726 68774 1 tig00000005 NODE_2_length_68774_cov_97.0195_ID_37066
tig00000005 Circularized: yes
tig00000004 Trying to circularize. Has nucmer hits to check...
tig00000004 No matches to SPAdes circular contigs
tig00000004 Could not circularize using matches to SPAdes circular contigs
tig00000004 Circularizing using this pair of nucmer matches to SPAdes contig:
tig00000004 1 49612 16420 66030 49612 49611 99.99 117048 66030 1 tig00000004 NODE_4_length_66030_cov_80.4157_ID_37070
tig00000004 81423 117048 1 35629 35626 35629 99.99 117048 66030 1 tig00000004 NODE_4_length_66030_cov_80.4157_ID_37070
tig00000004 Circularized: yes
tig00000006 Trying to circularize. Has nucmer hits to check...
tig00000006 No matches to SPAdes circular contigs
tig00000006 Could not circularize using matches to SPAdes circular contigs
tig00000006 Circularizing using this pair of nucmer matches to SPAdes contig:
tig00000006 3332 63098 59767 1 59767 59767 100.00 91442 69519 1 tig00000006 NODE_1_length_69519_cov_84.964_ID_37064
tig00000006 61023 91442 69519 39100 30420 30420 100.00 91442 69519 1 tig00000006 NODE_1_length_69519_cov_84.964_ID_37064
tig00000006 Circularized: yes

Thank you very much in advance.

Spades could not be detected

Hi,

I am trying to use the CIrclator v1.2.1 in Ubuntu 16.04, but when I am running the progcheck, it says that the program can not detect spades.py. I have put the location of the spades.py in PATH, but to no avail.
I have also attempted to run the install-dependencies.sh script, but it still give me the same result.

Running "spades.py" and "python2 /path/to/spades.py" on its own was okay.
I have tried both v3.6.0 and 3.6.2 of the Spades assembler.

Here is the outcome of progcheck:

Circlator version: 1.2.1

External dependencies:
bwa 0.7.12 /usr/bin/bwa
nucmer 3.1 /usr/bin/nucmer
prodigal 2.6.3 /usr/local/bin/prodigal
samtools 1.3.1 /usr/local/bin/samtools
Didn't find spades in path. Looked for:spades.py

Python version:
3.5.2 (default, Jul 5 2016, 12:43:10)
[GCC 5.4.0 20160609]

Python dependencies:
bio_assembly_refinement 0.5.1 /usr/local/lib/python3.5/dist-packages/bio_assembly_refinement-0.5.1-py3.5.egg/bio_assembly_refinement/init.py
openpyxl 2.4.0-b1 /usr/local/lib/python3.5/dist-packages/openpyxl-2.4.0b1-py3.5.egg/openpyxl/init.py
pyfastaq 3.12.1 /usr/local/lib/python3.5/dist-packages/pyfastaq-3.12.1-py3.5.egg/pyfastaq/init.py
pymummer 0.7.1 /usr/local/lib/python3.5/dist-packages/pymummer-0.7.1-py3.5.egg/pymummer/init.py
pysam 0.8.3 /usr/local/lib/python3.5/dist-packages/pysam-0.8.3-py3.5-linux-x86_64.egg/pysam/init.py

And this is outcome of "python2 /path/to/spades.py":

SPAdes genome assembler v.3.6.0

Usage: ./build/SPAdes-3.6.0-Linux/bin/spades.py [options] -o <output_dir>

Basic options:
-o <output_dir> directory to store all the resulting files (required)
--sc this flag is required for MDA (single-cell) data
--iontorrent this flag is required for IonTorrent data
--test runs SPAdes on toy dataset
-h/--help prints this usage message

Input data:
--12 file with interlaced forward and reverse paired-end reads
-1 file with forward paired-end reads
-2 file with reverse paired-end reads
-s file with unpaired reads
--pe<#>-12 file with interlaced reads for paired-end library number <#> (<#> = 1,2,3,4,5)
--pe<#>-1 file with forward reads for paired-end library number <#> (<#> = 1,2,3,4,5)
--pe<#>-2 file with reverse reads for paired-end library number <#> (<#> = 1,2,3,4,5)
--pe<#>-s file with unpaired reads for paired-end library number <#> (<#> = 1,2,3,4,5)
--pe<#>- orientation of reads for paired-end library number <#> (<#> = 1,2,3,4,5; = fr, rf, ff)
--s<#> file with unpaired reads for single reads library number <#> (<#> = 1,2,3,4,5)
--mp<#>-12 file with interlaced reads for mate-pair library number <#> (<#> = 1,2,3,4,5)
--mp<#>-1 file with forward reads for mate-pair library number <#> (<#> = 1,2,3,4,5)
--mp<#>-2 file with reverse reads for mate-pair library number <#> (<#> = 1,2,3,4,5)
--mp<#>-s file with unpaired reads for mate-pair library number <#> (<#> = 1,2,3,4,5)
--mp<#>- orientation of reads for mate-pair library number <#> (<#> = 1,2,3,4,5; = fr, rf, ff)
--hqmp<#>-12 file with interlaced reads for high-quality mate-pair library number <#> (<#> = 1,2,3,4,5)
--hqmp<#>-1 file with forward reads for high-quality mate-pair library number <#> (<#> = 1,2,3,4,5)
--hqmp<#>-2 file with reverse reads for high-quality mate-pair library number <#> (<#> = 1,2,3,4,5)
--hqmp<#>-s file with unpaired reads for high-quality mate-pair library number <#> (<#> = 1,2,3,4,5)
--hqmp<#>- orientation of reads for high-quality mate-pair library number <#> (<#> = 1,2,3,4,5; = fr, rf, ff)
--nxmate<#>-1 file with forward reads for Lucigen NxMate library number <#> (<#> = 1,2,3,4,5)
--nxmate<#>-2 file with reverse reads for Lucigen NxMate library number <#> (<#> = 1,2,3,4,5)
--sanger file with Sanger reads
--pacbio file with PacBio reads
--nanopore file with Nanopore reads
--trusted-contigs file with trusted contigs
--untrusted-contigs file with untrusted contigs

Pipeline options:
--only-error-correction runs only read error correction (without assembling)
--only-assembler runs only assembling (without read error correction)
--careful tries to reduce number of mismatches and short indels
--continue continue run from the last available check-point
--restart-from restart run with updated options and from the specified check-point ('ec', 'as', 'k', 'mc')
--disable-gzip-output forces error correction not to compress the corrected reads
--disable-rr disables repeat resolution stage of assembling

Advanced options:
--dataset file with dataset description in YAML format
-t/--threads number of threads
[default: 16]
-m/--memory RAM limit for SPAdes in Gb (terminates if exceeded)
[default: 250]
--tmp-dir directory for temporary files
[default: <output_dir>/tmp]
-k <int,int,...> comma-separated list of k-mer sizes (must be odd and
less than 128) [default: 'auto']
--cov-cutoff coverage cutoff value (a positive float number, or 'auto', or 'off') [default: 'off']
--phred-offset <33 or 64> PHRED quality offset in the input reads (33 or 64)
[default: auto-detect]

Can you please advice on this matter?
Thanks

samtools command error

I'm trying to run a "circlator all assembly.fasta reads output_directory" and got an error a few minutes after the run begins:

bam_sort] Use -T PREFIX / -o FILE to specify temporary and final output files
Usage: samtools sort [options...] [in.bam]
Options:
-l INT Set compression level, from 0 (uncompressed) to 9 (best)
-m INT Set maximum memory per thread; suffix K/M/G recognized [768M]
-n Sort by read name
-o FILE Write final output to FILE rather than standard output
-T PREFIX Write temporary files to PREFIX.nnnn.bam
-@, --threads INT
Set number of sorting and compression threads [1]
--input-fmt-option OPT[=VAL]
Specify a single input file format option in the form
of OPTION or OPTION=VALUE
-O, --output-fmt FORMAT[,OPT[=VAL]]...
Specify output format (SAM, BAM, CRAM)
--output-fmt-option OPT[=VAL]
Specify a single output file format option in the form
of OPTION or OPTION=VALUE
--reference FILE
Reference sequence FASTA FILE [null]

I believe this error occurs because the command is missing a '-o'. I tried to run this command (with -o added) myself and it works.

Thanks!

Include nucmer (or similar) option for fixstart

Hi!

Would it be possible to include the option to use a nucleotide fasta that should be searched for explicitly rather than translated with promer for the organisms that does not have the genome start defined by the beginning of a certain CDS?

Best wishes,
Kaisa

SPAdes 3.7.1 recommendation

WARNING: SPAdes version 3.9.0 is being used. It will work, but better results are usually obtained from Circlator using SPAdes version 3.7.1. Although 3.7.1 is not the latest version, we recommend it for Circlator.

Do you have any thoughts as to why SPAdes 3.7.1 performs better than 3.9.0 with Circlator?
cc Anton Korobeynikov @asl

@ilovezfs and I maintain the circlator package for Homebrew/Science, and we'd prefer to use SPAdes 3.9.0 with Circlator if possible.

Error running circlator 0.14.0 to reproduce results from published article

Dear Author,

I am trying to reproduce the results published in your article. After installing circlator, and all the other dependencies i get a error:

 /exports/sasc/sajvanderzeeuw/circlator-0.14.0/build/scripts-3.4/circlator all hgap_assembly.fasta corrected_reads.fasta circlator_0.14                                                                                                              Traceback (most recent call last):
  File "/home/sajvanderzeeuw/.virtualenvs/circlator/lib/python3.4/site-packages/pyfastaq-3.11.1-py3.4.egg/pyfastaq/utils.py", line 21, in open_file_read
FileNotFoundError: [Errno 2] No such file or directory: '03.assemble/contigs.fastg'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/exports/sasc/sajvanderzeeuw/circlator-0.14.0/build/scripts-3.4/circlator", line 52, in <module>
    exec('circlator.tasks.' + task + '.run()')
  File "<string>", line 1, in <module>
  File "/home/sajvanderzeeuw/.virtualenvs/circlator/lib/python3.4/site-packages/circlator-0.14.0-py3.4.egg/circlator/tasks/all.py", line 138, in run
    m.run()
  File "/home/sajvanderzeeuw/.virtualenvs/circlator/lib/python3.4/site-packages/circlator-0.14.0-py3.4.egg/circlator/merge.py", line 681, in run
    self._circularise_contigs(nucmer_hits)
  File "/home/sajvanderzeeuw/.virtualenvs/circlator/lib/python3.4/site-packages/circlator-0.14.0-py3.4.egg/circlator/merge.py", line 318, in _circularise_contigs
    called_as_circular_by_spades = self._get_spades_circular_nodes(reassembly_fastg)
  File "/home/sajvanderzeeuw/.virtualenvs/circlator/lib/python3.4/site-packages/circlator-0.14.0-py3.4.egg/circlator/merge.py", line 609, in _get_spades_circular_nodes
    names = set([x.id.rstrip(';') for x in seq_reader if ':' in x.id])
  File "/home/sajvanderzeeuw/.virtualenvs/circlator/lib/python3.4/site-packages/circlator-0.14.0-py3.4.egg/circlator/merge.py", line 609, in <listcomp>
    names = set([x.id.rstrip(';') for x in seq_reader if ':' in x.id])
  File "/home/sajvanderzeeuw/.virtualenvs/circlator/lib/python3.4/site-packages/pyfastaq-3.11.1-py3.4.egg/pyfastaq/sequences.py", line 35, in file_reader
  File "/home/sajvanderzeeuw/.virtualenvs/circlator/lib/python3.4/site-packages/pyfastaq-3.11.1-py3.4.egg/pyfastaq/utils.py", line 23, in open_file_read
pyfastaq.utils.Error: Error opening for reading file '03.assemble/contigs.fastg'

This is off course a very obvious error because the tools expect a file which contains a wrong name. This is a known bug in your 0.14.0 release?

Also i tried 1 of the newest releases 1.1.3. But here i cant reproduce your plasmodium results. It gives me that contig 96 is not circular.

See the folowing diff:

diff 04.merge.circularise.log ../test_circular_with_dnaAfile/04.merge.circularise.log
1a2
> [merge circularised]  unitig_97|quiver        0       0       0       0
3d3
< [merge circularised]  unitig_97|quiver        0       0       1       1

When i check if all dependencies are there it says all test succeeded. This is the point i'm stuck at.

Basically the question i have is:

How can i reproduce the findings of your published article: http://genomebiology.biomedcentral.com/articles/10.1186/s13059-015-0849-0

And which options are nice to take into account for circularizing genomes?

Thanks in advance!

Best regards,

Sander

error encountered when trying to install from the source

Dear Martin,

I downloaded the most recent version ("circlator-1.5.1") of the circlator source code and tried to run the installation from the source. However, I encountered the following error immediately when running the "python3 setup.py test" command inside the uncompressed directory. Could you help to figure out what might be the cause of this error? Thanks in advance!

Best,
Jia-Xing

#########

python3 setup.py test

running test
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/setuptools/dist.py", line 414, in fetch_build_egg
cmd = self._egg_fetcher
AttributeError: 'Distribution' object has no attribute '_egg_fetcher'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "setup.py", line 31, in
'License :: OSI Approved :: GNU General Public License v3 (GPLv3)',
File "/usr/lib64/python3.4/distutils/core.py", line 148, in setup
dist.run_commands()
File "/usr/lib64/python3.4/distutils/dist.py", line 955, in run_commands
self.run_command(cmd)
File "/usr/lib64/python3.4/distutils/dist.py", line 974, in run_command
cmd_obj.run()
File "/usr/lib/python2.7/site-packages/setuptools/command/test.py", line 199, in run
installed_dists = self.install_dists(self.distribution)
File "/usr/lib/python2.7/site-packages/setuptools/command/test.py", line 194, in install_dists
ir_d = dist.fetch_build_eggs(dist.install_requires or [])
File "/usr/lib/python2.7/site-packages/setuptools/dist.py", line 373, in fetch_build_eggs
replace_conflicting=True,
File "/usr/lib/python2.7/site-packages/pkg_resources/init.py", line 851, in resolve
dist = best[req.key] = env.best_match(req, ws, installer)
File "/usr/lib/python2.7/site-packages/pkg_resources/init.py", line 1123, in best_match
return self.obtain(req, installer)
File "/usr/lib/python2.7/site-packages/pkg_resources/init.py", line 1135, in obtain
return installer(requirement)
File "/usr/lib/python2.7/site-packages/setuptools/dist.py", line 417, in fetch_build_egg
from setuptools.command.easy_install import easy_install
File "/usr/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 52, in
from setuptools.package_index import (
File "/usr/lib/python2.7/site-packages/setuptools/package_index.py", line 18, in
from six.moves import urllib, http_client, configparser, map
File "/usr/lib/python2.7/site-packages/six.py", line 92, in get
result = self._resolve()
File "/usr/lib/python2.7/site-packages/six.py", line 115, in _resolve
return _import_module(self.mod)
File "/usr/lib/python2.7/site-packages/six.py", line 82, in _import_module
import(name)
File "/usr/lib/python2.7/site-packages/http/init.py", line 7, in
raise ImportError('This package should not be accessible on Python 3. '
ImportError: This package should not be accessible on Python 3. Either you are trying to run from the python-future src folder or your installation of python-future is corrupted.

Add --verbose to the other subcommands

When I run commands like

circlator fixstart --genes_fa dnaA.aa.fa pilon.fasta pilon.fixstart

it prints absolutely nothing.

This makes me think something is wrong and induces CTRL-C anxiety, only to be presented with a Python backtrace :-)

I can see it is calling syscall.run('bash ' + script, verbose=self.verbose) but I am not able to set the verbosity from the command line.

Can circlator work with an assembly with ambiguous bases?

Hi,
I merged multiple nanopore contigs using illumina data but I have some ambiguous bases and Ns and some gaps in the final contig. I want to circularize this final contig with the original nanopore data. Will circlator work with this ambiguous bases?

Thanks!

contig.fastg went missing because of SPAdes 3.6.1 output changes

Hi,

circlator is crashing because contigs.fastg went missing because in SPAdes 3.6.1 contigs.fastg files are not supported anymore (See below changelog SPAdes 3.6,1). Could you please look into this.

Traceback (most recent call last):
File "/.linuxbrew/opt/python3/lib/python3.5/site-packages/pyfastaq/utils.py", line 21, in open_file_read
f = open(filename)
FileNotFoundError: [Errno 2] No such file or directory: '03.assemble/contigs.fastg'

"SPAdes Genome Assembler changelog
SPAdes 3.6.1, 4 October 2015

CHANGE: No misleading FASTG files, only assembly graph is saved in FASTG format. "

Thank you in advance,
Best regards,
Christian

Option to change assembler options

Add an option that will let the user pass arbitrary options to the assembler.

Spades version detection bit wonky?

It prints it as `.7.0` instead of `3.7.0` below?

I'm using circlator version 1.2.0

Using bwa version 0.7.13 as /bio/linuxbrew/bin/bwa
Using nucmer version 3.1 as /bio/linuxbrew/bin/nucmer
Using prodigal version 2.6.2 as /bio/linuxbrew/bin/prodigal
Using samtools version 1.3 as /bio/linuxbrew/bin/samtools
Using spades version .7.0 as /bio/linuxbrew/bin/spades.py

PS. they only added a --version flag in 3.7.0

sanger-pathogens / circlator Goto Github PK

circlator's People

Contributors

Stargazers

Watchers

Forkers

circlator's Issues

reading input file "p.ntref" of length 6680340

construct suffix tree for sequence of length 6680340

(maximum reference length is 536870908)

(maximum query length is 4294967295)

process 66803 characters per dot

CONSTRUCTIONTIME /home/linuxbrew/.linuxbrew/Cellar/mummer/3.23_1/mummer p.ntref 2.84

reading input file "/home/zong/Desktop/666/96200/circlator/spades/03.assemble/contigs.fasta" of length 1067392

matching query-file "/home/zong/Desktop/666/96200/circlator/spades/03.assemble/contigs.fasta"

against subject-file "p.ntref"

COMPLETETIME /home/linuxbrew/.linuxbrew/Cellar/mummer/3.23_1/mummer p.ntref 3.81

SPACE /home/linuxbrew/.linuxbrew/Cellar/mummer/3.23_1/mummer p.ntref 7.51

reading input file "p.ntref" of length 6669533

construct suffix tree for sequence of length 6669533

(maximum reference length is 536870908)

(maximum query length is 4294967295)

process 66695 characters per dot

CONSTRUCTIONTIME /home/linuxbrew/.linuxbrew/Cellar/mummer/3.23_1/mummer p.ntref 2.69

reading input file "/home/zong/Desktop/666/96200/circlator/spades/04.merge.merge.iter.2.assembly/contigs.fasta" of length 1188473

matching query-file "/home/zong/Desktop/666/96200/circlator/spades/04.merge.merge.iter.2.assembly/contigs.fasta"

against subject-file "p.ntref"

COMPLETETIME /home/linuxbrew/.linuxbrew/Cellar/mummer/3.23_1/mummer p.ntref 3.77

SPACE /home/linuxbrew/.linuxbrew/Cellar/mummer/3.23_1/mummer p.ntref 7.62

pysam: no cython available - using pre-compiled C

pysam: htslib mode is shared

pysam: HTSLIB_CONFIGURE_OPTIONS=None

pysam: htslib configure options: None

pysam: htslib_config LIBS=-lz -lm -lbz2 -llzma

pysam: htslib_config PLATFORM=Linux

pysam: htslib_config LDFLAGS=

pysam: config_option: ENABLE_PLUGINS=0

pysam: config_option: HAVE_COMMONCRYPTO=0

pysam: config_option: HAVE_GMTIME_R=0

pysam: config_option: HAVE_HMAC=0

pysam: config_option: HAVE_IRODS=0

pysam: config_option: HAVE_LIBCURL=0

pysam: config_option: HAVE_MMAP=0

It prints it as .7.0 instead of 3.7.0 below?

Recommend Projects

Recommend Topics

Recommend Org

It prints it as `.7.0` instead of `3.7.0` below?