problem with pfam.hmm formatting? about checkm

[Xiaojun928]

It is my fault to miss an older version of HMMER. As you guessed, v3.1b2 is not active one. After I replace the old version, CheckM works! Thanks a lot~

[wwood]

Thanks for the detailed bug report @jfroula - @dparks1134 is on holidays this week, I imagine he'll get back to you next week.

[donovan-h-parks]

This problem occurs when using HMMER < 3.1b1. I've updated the dependency list for CheckM to indicate that it requires HMMER >=3.1b1. Thank you for bringing this to my attention Jeff.

[jfroula]

Seems to work now with updated hmmer.
Thanks again.
-jeff

On Tue, Oct 7, 2014 at 3:38 PM, Donovan Parks [email protected]
wrote:

This problem occurs when using HMMER < 3.1b1. I've updated the dependency
list for CheckM to indicate that it requires HMMER >=3.1b1. Thank you for
bringing this to my attention Jeff.

—
Reply to this email directly or view it on GitHub
#8 (comment).

[jmwhitha]

I think I'm having a similar problem. Please help.

checkm lineage_wf -f ./CheckM.txt -t 12 -x fa ./ ./94444

---

[CheckM - tree] Placing bins in reference genome tree.

---

Identifying marker genes in 42 bins with 12 threads:
Process Process-10:sing 0 of 42 (0.00%) bins.
Traceback (most recent call last):
File "/usr/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
self.run()
File "/usr/lib/python2.7/multiprocessing/process.py", line 114, in run
self._target(_self._args, *_self._kwargs)
File "/usr/local/lib/python2.7/dist-packages/checkm/markerGeneFinder.py", line 122, in __processBin
hmmModelFile = markerSetParser.createHmmModelFile(binId, markerFile)
File "/usr/local/lib/python2.7/dist-packages/checkm/markerSets.py", line 318, in createHmmModelFile
markerFileType = self.markerFileType(markerFile)
File "/usr/local/lib/python2.7/dist-packages/checkm/markerSets.py", line 418, in markerFileType
with open(markerFile, 'r') as f:
IOError: [Errno 2] No such file or directory: u'/usr/local/metabat/MetaBatbinning/checkMDATA/hmms/phylo.hmm'
Process Process-4:
Traceback (most recent call last):
File "/usr/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
self.run()
File "/usr/lib/python2.7/multiprocessing/process.py", line 114, in run
self._target(_self._args, *_self._kwargs)
File "/usr/local/lib/python2.7/dist-packages/checkm/markerGeneFinder.py", line 122, in __processBin
hmmModelFile = markerSetParser.createHmmModelFile(binId, markerFile)
File "/usr/local/lib/python2.7/dist-packages/checkm/markerSets.py", line 318, in createHmmModelFile
markerFileType = self.markerFileType(markerFile)
File "/usr/local/lib/python2.7/dist-packages/checkm/markerSets.py", line 418, in markerFileType
with open(markerFile, 'r') as f:
IOError: [Errno 2] No such file or directory: u'/usr/local/metabat/MetaBatbinning/checkMDATA/hmms/phylo.hmm'
Process Process-8:
Traceback (most recent call last):
File "/usr/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
self.run()
File "/usr/lib/python2.7/multiprocessing/process.py", line 114, in run
self._target(_self._args, *_self._kwargs)
File "/usr/local/lib/python2.7/dist-packages/checkm/markerGeneFinder.py", line 122, in __processBin
hmmModelFile = markerSetParser.createHmmModelFile(binId, markerFile)
File "/usr/local/lib/python2.7/dist-packages/checkm/markerSets.py", line 318, in createHmmModelFile
markerFileType = self.markerFileType(markerFile)
File "/usr/local/lib/python2.7/dist-packages/checkm/markerSets.py", line 418, in markerFileType
with open(markerFile, 'r') as f:
IOError: [Errno 2] No such file or directory: u'/usr/local/metabat/MetaBatbinning/checkMDATA/hmms/phylo.hmm'
Process Process-11:
Traceback (most recent call last):
File "/usr/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
self.run()
File "/usr/lib/python2.7/multiprocessing/process.py", line 114, in run
self._target(_self._args, *_self._kwargs)
File "/usr/local/lib/python2.7/dist-packages/checkm/markerGeneFinder.py", line 122, in __processBin
hmmModelFile = markerSetParser.createHmmModelFile(binId, markerFile)
File "/usr/local/lib/python2.7/dist-packages/checkm/markerSets.py", line 318, in createHmmModelFile
markerFileType = self.markerFileType(markerFile)
File "/usr/local/lib/python2.7/dist-packages/checkm/markerSets.py", line 418, in markerFileType
with open(markerFile, 'r') as f:
IOError: [Errno 2] No such file or directory: u'/usr/local/metabat/MetaBatbinning/checkMDATA/hmms/phylo.hmm'
Process Process-6:
Traceback (most recent call last):
File "/usr/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
self.run()
File "/usr/lib/python2.7/multiprocessing/process.py", line 114, in run
self._target(_self._args, *_self._kwargs)
File "/usr/local/lib/python2.7/dist-packages/checkm/markerGeneFinder.py", line 122, in __processBin
hmmModelFile = markerSetParser.createHmmModelFile(binId, markerFile)
File "/usr/local/lib/python2.7/dist-packages/checkm/markerSets.py", line 318, in createHmmModelFile
markerFileType = self.markerFileType(markerFile)
File "/usr/local/lib/python2.7/dist-packages/checkm/markerSets.py", line 418, in markerFileType
with open(markerFile, 'r') as f:
IOError: [Errno 2] No such file or directory: u'/usr/local/metabat/MetaBatbinning/checkMDATA/hmms/phylo.hmm'
Process Process-9:
Traceback (most recent call last):
File "/usr/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
self.run()
File "/usr/lib/python2.7/multiprocessing/process.py", line 114, in run
self._target(_self._args, *_self._kwargs)
File "/usr/local/lib/python2.7/dist-packages/checkm/markerGeneFinder.py", line 122, in __processBin
hmmModelFile = markerSetParser.createHmmModelFile(binId, markerFile)
File "/usr/local/lib/python2.7/dist-packages/checkm/markerSets.py", line 318, in createHmmModelFile
markerFileType = self.markerFileType(markerFile)
File "/usr/local/lib/python2.7/dist-packages/checkm/markerSets.py", line 418, in markerFileType
with open(markerFile, 'r') as f:
IOError: [Errno 2] No such file or directory: u'/usr/local/metabat/MetaBatbinning/checkMDATA/hmms/phylo.hmm'
Process Process-12:
Traceback (most recent call last):
File "/usr/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
self.run()
File "/usr/lib/python2.7/multiprocessing/process.py", line 114, in run
self._target(_self._args, *_self._kwargs)
File "/usr/local/lib/python2.7/dist-packages/checkm/markerGeneFinder.py", line 122, in __processBin
hmmModelFile = markerSetParser.createHmmModelFile(binId, markerFile)
File "/usr/local/lib/python2.7/dist-packages/checkm/markerSets.py", line 318, in createHmmModelFile
markerFileType = self.markerFileType(markerFile)
File "/usr/local/lib/python2.7/dist-packages/checkm/markerSets.py", line 418, in markerFileType
with open(markerFile, 'r') as f:
IOError: [Errno 2] No such file or directory: u'/usr/local/metabat/MetaBatbinning/checkMDATA/hmms/phylo.hmm'
Process Process-7:
Traceback (most recent call last):
File "/usr/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
self.run()
File "/usr/lib/python2.7/multiprocessing/process.py", line 114, in run
self._target(_self._args, *_self._kwargs)
File "/usr/local/lib/python2.7/dist-packages/checkm/markerGeneFinder.py", line 122, in __processBin
hmmModelFile = markerSetParser.createHmmModelFile(binId, markerFile)
File "/usr/local/lib/python2.7/dist-packages/checkm/markerSets.py", line 318, in createHmmModelFile
markerFileType = self.markerFileType(markerFile)
File "/usr/local/lib/python2.7/dist-packages/checkm/markerSets.py", line 418, in markerFileType
with open(markerFile, 'r') as f:
IOError: [Errno 2] No such file or directory: u'/usr/local/metabat/MetaBatbinning/checkMDATA/hmms/phylo.hmm'
Process Process-3:
Traceback (most recent call last):
File "/usr/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
self.run()
File "/usr/lib/python2.7/multiprocessing/process.py", line 114, in run
self._target(_self._args, *_self._kwargs)
File "/usr/local/lib/python2.7/dist-packages/checkm/markerGeneFinder.py", line 122, in __processBin
hmmModelFile = markerSetParser.createHmmModelFile(binId, markerFile)
File "/usr/local/lib/python2.7/dist-packages/checkm/markerSets.py", line 318, in createHmmModelFile
markerFileType = self.markerFileType(markerFile)
File "/usr/local/lib/python2.7/dist-packages/checkm/markerSets.py", line 418, in markerFileType
with open(markerFile, 'r') as f:
IOError: [Errno 2] No such file or directory: u'/usr/local/metabat/MetaBatbinning/checkMDATA/hmms/phylo.hmm'
Process Process-5:
Traceback (most recent call last):
File "/usr/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
self.run()
File "/usr/lib/python2.7/multiprocessing/process.py", line 114, in run
self._target(_self._args, *_self._kwargs)
File "/usr/local/lib/python2.7/dist-packages/checkm/markerGeneFinder.py", line 122, in __processBin
hmmModelFile = markerSetParser.createHmmModelFile(binId, markerFile)
File "/usr/local/lib/python2.7/dist-packages/checkm/markerSets.py", line 318, in createHmmModelFile
markerFileType = self.markerFileType(markerFile)
File "/usr/local/lib/python2.7/dist-packages/checkm/markerSets.py", line 418, in markerFileType
with open(markerFile, 'r') as f:
IOError: [Errno 2] No such file or directory: u'/usr/local/metabat/MetaBatbinning/checkMDATA/hmms/phylo.hmm'
Process Process-13:
Traceback (most recent call last):
File "/usr/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
self.run()
File "/usr/lib/python2.7/multiprocessing/process.py", line 114, in run
self._target(_self._args, *_self._kwargs)
File "/usr/local/lib/python2.7/dist-packages/checkm/markerGeneFinder.py", line 122, in __processBin
hmmModelFile = markerSetParser.createHmmModelFile(binId, markerFile)
File "/usr/local/lib/python2.7/dist-packages/checkm/markerSets.py", line 318, in createHmmModelFile
markerFileType = self.markerFileType(markerFile)
File "/usr/local/lib/python2.7/dist-packages/checkm/markerSets.py", line 418, in markerFileType
with open(markerFile, 'r') as f:
IOError: [Errno 2] No such file or directory: u'/usr/local/metabat/MetaBatbinning/checkMDATA/hmms/phylo.hmm'
Process Process-2:
Traceback (most recent call last):
File "/usr/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
self.run()
File "/usr/lib/python2.7/multiprocessing/process.py", line 114, in run
self._target(_self._args, *_self._kwargs)
File "/usr/local/lib/python2.7/dist-packages/checkm/markerGeneFinder.py", line 122, in __processBin
hmmModelFile = markerSetParser.createHmmModelFile(binId, markerFile)
File "/usr/local/lib/python2.7/dist-packages/checkm/markerSets.py", line 318, in createHmmModelFile
markerFileType = self.markerFileType(markerFile)
File "/usr/local/lib/python2.7/dist-packages/checkm/markerSets.py", line 418, in markerFileType
with open(markerFile, 'r') as f:
IOError: [Errno 2] No such file or directory: u'/usr/local/metabat/MetaBatbinning/checkMDATA/hmms/phylo.hmm'

Saving HMM info to file.

Calculating genome statistics for 42 bins with 12 threads:
Finished processing 42 of 42 (100.00%) bins.

Extracting marker genes to align.
[Error] Models must be parsed before identifying HMM hits.

Unexpected error: <type 'exceptions.AttributeError'>
Traceback (most recent call last):
File "/usr/local/bin/checkm", line 712, in
checkmParser.parseOptions(args)
File "/usr/local/lib/python2.7/dist-packages/checkm/main.py", line 1244, in parseOptions
self.tree(options)
File "/usr/local/lib/python2.7/dist-packages/checkm/main.py", line 156, in tree
os.path.join(options.out_folder, 'storage', 'tree')
File "/usr/local/lib/python2.7/dist-packages/checkm/hmmerAligner.py", line 104, in makeAlignmentToPhyloMarkers
resultsParser.parseBinHits(outDir, hmmTableFile, False, bIgnoreThresholds, evalueThreshold, lengthThreshold)
File "/usr/local/lib/python2.7/dist-packages/checkm/main.py", line 1205, in parseOptions
if options.bVerbose:
AttributeError: 'Namespace' object has no attribute 'bVerbose'

[Xiaojun928]

As you discussed above, HMMER >=3.1b1 works, but in my case, I installed HMMER 3.1b2 and added prodigal, hmmer, and pplacer to PATH. I got the same error like @jfroula
My version of CheckM is v1.0.11
Could you please do me some help? Many thanks.

~/anaconda2/bin/checkm lineage_wf ./test_data/ Results

---

[CheckM - tree] Placing bins in reference genome tree.

---

Identifying marker genes in 1 bins with 1 threads:
Finished processing 0 of 1 (0.00%) bins.
Error: Unrecognized format, trying to open hmm file /tmp/0a97cb01-5f8f-4574-b3ac-ce4ceff69da6 for reading.

Finished processing 1 of 1 (100.00%) bins.

Saving HMM info to file.

Calculating genome statistics for 1 bins with 1 threads:
Finished processing 1 of 1 (100.00%) bins.

Extracting marker genes to align.
Parsing HMM hits to marker genes:
[Errno 2] No such file or directory: 'Results/bins/637000110/hmmer.tree.txt'

Extracting 43 HMMs with 1 threads:
Finished extracting 0 of 43 (0.00%) HMMs.
Error: File /home-user/xjwang/software/checkm_data_2015_01_16/hmms/phylo.hmm does not appear to be in a recognized HMM format.

Finished extracting 1 of 43 (2.33%) HMMs.

Error: File /home-user/xjwang/software/checkm_data_2015_01_16/hmms/phylo.hmm does not appear to be in a recognized HMM format.
... ...
Finished extracting 43 of 43 (100.00%) HMMs.
Aligning 43 marker genes with 1 threads:
Finished aligning 43 of 43 (100.00%) marker genes.

Reading marker alignment files.
Concatenating alignments.
No genomes were identified that could be placed in the reference genome tree.

{ Current stage: 0:00:18.546 || Total: 0:00:18.546 }

---

[CheckM - lineage_set] Inferring lineage-specific marker sets.

---

Reading HMM info from file.
Parsing HMM hits to marker genes:
[Errno 2] No such file or directory: 'Results/bins/637000110/hmmer.tree.txt'

Determining marker sets for each genome bin.
Finished processing 1 of 1 (100.00%) bins (current: 637000110).

Marker set written to: Results/lineage.ms

{ Current stage: 0:00:00.812 || Total: 0:00:19.358 }

---

[CheckM - analyze] Identifying marker genes in bins.

---

Identifying marker genes in 1 bins with 1 threads:
Finished processing 0 of 1 (0.00%) bins.
Error: File /home-user/xjwang/software/checkm_data_2015_01_16/hmms/checkm.hmm does not appear to be in a recognized HMM format.

Error: File /tmp/6ebe369b-e9a1-40fb-a026-4e9877277cd2 does not appear to be in a recognized HMM format.

Error: Unrecognized format, trying to open hmm file /tmp/6ebe369b-e9a1-40fb-a026-4e9877277cd2 for reading.

Finished processing 1 of 1 (100.00%) bins.

Saving HMM info to file.

{ Current stage: 0:00:00.435 || Total: 0:00:19.794 }

Parsing HMM hits to marker genes:
[Errno 2] No such file or directory: 'Results/bins/637000110/hmmer.analyze.txt'

Aligning marker genes with multiple hits in a single bin:
Finished processing 1 of 1 (100.00%) bins.

{ Current stage: 0:00:00.053 || Total: 0:00:19.847 }

Calculating genome statistics for 1 bins with 1 threads:
Finished processing 1 of 1 (100.00%) bins.

{ Current stage: 0:00:00.210 || Total: 0:00:20.058 }

---

[CheckM - qa] Tabulating genome statistics.

---

Calculating AAI between multi-copy marker genes.

Reading HMM info from file.
Parsing HMM hits to marker genes:
[Errno 2] No such file or directory: 'Results/bins/637000110/hmmer.analyze.txt'

---

Bin Id Marker lineage # genomes # markers # marker sets 0 1 2 3 4 5+ Completeness Contamination Strain heterogeneity

637000110 root (UID1) 5656 56 24 56 0 0 0 0 0 0.00 0.00 0.00

{ Current stage: 0:00:00.006 || Total: 0:00:20.064 }

[donovan-h-parks]

Hello. I'm not sure what the issue is. I can confirm that we are using HMMER 3.1b2 on our machines with CheckM and it works without issue. Can you confirm that hmmsearch v3.1b2 is the active version on your path. You can check the version with "hmmsearch -h".

You could also try re-indexing the phylo.hmm file:

hmmfetch --index phylo.hmm

This is at /home-user/xjwang/software/checkm_data_2015_01_16/hmms/ on your system. I don't think this is the issue, but worth a try.