AttributeError in Unit tests step 1 about checkm

[wwood]

@dparks1134 is on holidays at the moment, but if I was to have a stab at this I might think that you have 2 versions of checkm installed. The new checkm is what you are calling when you run checkm test, but that checkm is loading the old version of python when it imports (and options.pplacer is a relatively new addition). Possible?

[LouieF]

@wwood I don't think so, I just installed one checkm and I have never used it before. Thank you for reply :)

[wwood]

Ah, I see. From

https://github.com/Ecogenomics/CheckM/wiki/Installation#unit-tests

Note: this unit test is currently out-of-sync will some recent features. As such, it will fail during Step 3. However, this is sufficient to show that CheckM is install properly. The unit tests will be fixed in the next release.

So there it is, you don't seem to have much to worry about. I'm not sure you are at step 3, but I think you should be good to go and use checkm on real data, and the error you are encountering is an issue of syncing the tests with the code. Thanks for caring about software integrity enough to run the tests though.

[LouieF]

@wwood I tried real bins, but there are still errors.

yifan@yifan-VirtualBox:~/Documents/Working_directory/wd_for_checkm$ checkm lineage_wf -t 4  /home/yifan/Documents/Working_directory/wd_for_checkm /home/yifan/Documents/Working_directory/wd_for_checkm

*******************************************************************************
 [CheckM - tree] Placing bins in reference genome tree.
*******************************************************************************

[Warning] Output directory must be empty: /home/yifan/Documents/Working_directory/wd_for_checkm


Controlled exit resulting from an unrecoverable error or warning.
yifan@yifan-VirtualBox:~/Documents/Working_directory/wd_for_checkm$ checkm lineage_wf -t 4 /home/yifan/Documents/Working_directory/wd_for_checkm /home/yifan/Documents/Working_directory/wd_for_checkm/results

*******************************************************************************
[CheckM - tree] Placing bins in reference genome tree.
*******************************************************************************
Identifying marker genes in 3 bins with 4 threads:
Finished processing 3 of 3 (100.00%) bins.
Saving HMM info to file.

Calculating genome statistics for 3 bins with 4 threads:
Finished processing 3 of 3 (100.00%) bins.

Extracting marker genes to align.
Parsing HMM hits to marker genes:
Finished parsing hits for 3 of 3 (100.00%) bins.
Extracting 43 HMMs with 4 threads:
Finished extracting 43 of 43 (100.00%) HMMs.
Aligning 43 marker genes with 4 threads:
Finished aligning 43 of 43 (100.00%) marker genes.

Reading marker alignment files.
Concatenating alignments.
Placing 3 bins into the genome tree with pplacer (be patient).
Killed
Uncaught exception: Sys_error("/home/yifan/Documents/Working_directory/wd_for_checkm    /results/storage/tree/concatenated.pplacer.json: No such file or directory")
Fatal error: exception Sys_error("/home/yifan/Documents/Working_directory/wd_for_checkm/results/storage/tree/concatenated.pplacer.json: No such file or directory")

{ Current stage: 0:17:50.602 || Total: 0:17:50.602 }

*******************************************************************************
[CheckM - lineage_set] Inferring lineage-specific marker sets.
*******************************************************************************

Reading HMM info from file.
Parsing HMM hits to marker genes:
Finished parsing hits for 3 of 3 (100.00%) bins.

Determining marker sets for each genome bin.

Unexpected error: <type 'exceptions.IOError'>
Traceback (most recent call last):
File "/usr/local/bin/checkm", line 712, in <module>
checkmParser.parseOptions(args)
File "/usr/local/lib/python2.7/dist-packages/checkm/main.py", line 1241, in parseOptions
self.lineageSet(options)
File "/usr/local/lib/python2.7/dist-packages/checkm/main.py", line 226, in lineageSet
resultsParser, options.unique, options.multi)
File "/usr/local/lib/python2.7/dist-packages/checkm/treeParser.py", line 477, in getBinMarkerSets
tree = dendropy.Tree.get_from_path(treeFile, schema='newick', rooting="force-rooted", preserve_underscores=True)
File "/usr/local/lib/python2.7/dist-packages/dendropy/datamodel/basemodel.py", line 217, in get_from_path
with open(src, "r", newline=None) as fsrc:
File "/usr/local/lib/python2.7/dist-packages/dendropy/utility/filesys.py", line 60, in pre_py34_open
buffering=buffering)
IOError: [Errno 2] No such file or directory: '/home/yifan/Documents/Working_directory/wd_for_checkm/results/storage/tree/concatenated.tre'

[wwood]

On 19/12/15 11:07, LouieF wrote:

@wwood https://github.com/wwood I tried real bins, but there are
still errors.

|yifan@yifan-VirtualBox:~/Documents/Working_directory/wd_for_checkm$
checkm lineage_wf -t 4
/home/yifan/Documents/Working_directory/wd_for_checkm
/home/yifan/Documents/Working_directory/wd_for_checkm

---

[CheckM - tree] Placing bins in reference genome tree.

---

[Warning] Output directory must be empty:
/home/yifan/Documents/Working_directory/wd_for_checkm Controlled exit
resulting from an unrecoverable error or warning.|
|I take it you understand why this didn't work, the results directory
has to be empty.|
|yifan@yifan-VirtualBox:~/Documents/Working_directory/wd_for_checkm$
checkm lineage_wf -t 4
/home/yifan/Documents/Working_directory/wd_for_checkm
/home/yifan/Documents/Working_directory/wd_for_checkm/results

---

[CheckM - tree] Placing bins in reference genome tree.

---

Identifying marker genes in 3 bins with 4 threads: Finished processing
3 of 3 (100.00%) bins. Saving HMM info to file. Calculating genome
statistics for 3 bins with 4 threads: Finished processing 3 of 3
(100.00%) bins. Extracting marker genes to align. Parsing HMM hits to
marker genes: Finished parsing hits for 3 of 3 (100.00%) bins.
Extracting 43 HMMs with 4 threads: Finished extracting 43 of 43
(100.00%) HMMs. Aligning 43 marker genes with 4 threads: Finished
aligning 43 of 43 (100.00%) marker genes. Reading marker alignment
files. Concatenating alignments. Placing 3 bins into the genome tree
with pplacer (be patient). Killed|
|The error is right here, and is different to before. "Killed" indicates
that a subprocess (pplacer) was killed by the kernel, likely because
pplacer was asking for too much memory. Try upping the memory available
in the VM, perhaps?|

Hope that helps.

[LouieF]

@wwood That is awesome! I tried '--reduced_tree' flag and it works! Thanks a lot buddy!