arkadiy-garber / fegenie Goto Github PK
View Code? Open in Web Editor NEWHMM-based identification and categorization of iron genes and iron gene operons in genomes and metagenomes
License: GNU Affero General Public License v3.0
HMM-based identification and categorization of iron genes and iron gene operons in genomes and metagenomes
License: GNU Affero General Public License v3.0
Hi Arkadiy,
I am trying to use FeGenie to identify proteins involved in iron reduction.
My inputs are two amino acid fasta files from two metatranscritptome data. they are significant genes estimated by edgeR.
I ran the command:
(fegenie) sxxn@Seans-MacBook-Pro ~ % FeGenie.py -bin_dir /Volumes/SEAN/edgeR_DEG/ --orfs --skip -bin_ext fasta -out fegenie_out
then i got this:
Ok, proceeding with analysis!
All required arguments provided!
Traceback (most recent call last):
File "/Users/sxxn/miniconda3/envs/fegenie/bin/FeGenie.py", line 3020, in
main()
File "/Users/sxxn/miniconda3/envs/fegenie/bin/FeGenie.py", line 2744, in main
if args.hbm:
^^^^^^^^
AttributeError: 'Namespace' object has no attribute 'hbm'
and here is my input fasta:
4085_2
MRLTTKGRFAVTAMIDVALRQHAGPVTLAGIAERQKISLSYLEQLFGKLRRNQLVASTRGPGGGYTLAKPLAAVSVADIISAVDEPLDATSCGGRGNCHDDHPCMTHDLWMSLNARMHEYLSSVNLDHLVRQQGIKGCANEAQPVTIGKAPARRIPVMATA*
7_1
MKVTKIFTHGLLWVIIVMLVIPPGIMAQDTEETVQSAQFNEEELAQMLAPIALYPDSLIAEILMASTYPIEVVEAER
9_1
MKGMKIYIKGLSWVIIVMLMMPPGLMAQDSGQIEQPVKFNKEELAQMLAPIALYPDSLIAQILMASTYPIEVVEAERWIRKNKNLTGDELDSALQEKTWDPSVKSLCHFPDILYAMSEKLDQTTKLGDAFLSQQDEVMDTIQELRRKAQEQGNLTTTKEQKVIVEQETIYIEPANPEIIYVPAYDPLYVYGPWWYPAYP
Any idea I can fix this error?
Thanks,
Sean
Awesome product!
Many problems mentioned in the issues could be solved by rewriting the workflow using a proper manager. This would also properly parallelize tasks.
Best
Running FeGenie using -bams options with 4 metagenomes, each with their own single bam file. FeGenie proceeds through the mapping of the first set of contigs then crashes. This happens on the first file no matter what (I have put them in different orders) and each individual metagenome and bam file work fine when run singly with the -bam option.
processing... P1DNA.contigs.fa
Output depth matrix to geomics_bam_test_out/P1DNA.contigs.fa.depth
jgi_summarize_bam_contig_depths 2.15 (Bioconda) 2020-07-03T11:59:07
Output matrix to geomics_bam_test_out/P1DNA.contigs.fa.depth
0: Opening bam: /Users/gabrielle/Desktop/GenomeFun/FeGenie/Testing/BamFiles/P1DNA.bam
Processing bam files
Thread 0 finished: P1DNA.bam with 3840282 reads and 2255380 readsWellMapped
Creating depth matrix file: geomics_bam_test_out/P1DNA.contigs.fa.depth
Closing most bam files
Closing last bam file
Finished
processing... P1DNA.contigs.fa
Traceback (most recent call last):
File "/Users/gabrielle/Desktop/GenomeFun/FeGenie/FeGenie.py", line 2707, in <module>
main()
File "/Users/gabrielle/Desktop/GenomeFun/FeGenie/FeGenie.py", line 2341, in main
Dict[cell][process].append(float(depthDict[cell][contig]))
ValueError: could not convert string to float: 'EMPTY'
Note--this was happening last week as well but I was worried it was perhaps because of how I had gotten the single -bam option to work by the metabat install, so I did not post then. Attaching my bam location file just in case.
Bamlocations2.txt
Hi,
When I run the program providing a reference database it throws an error which I cannot figure out how to solve. Any help is appreciated!
Cheers,
Christoph
/Final processing of output
Traceback (most recent call last):
File "/home/ckeuschn/miniconda3/envs/fegenie/bin/FeGenie.py", line 3020, in
main()
File "/home/ckeuschn/miniconda3/envs/fegenie/bin/FeGenie.py", line 2355, in main
ls[0] + "," + ls[1] + "," + str(idxDict[ls[2]]) + "," + ls[3] + "," + ls[4] + "," + ls[
^^^^^^^
UnboundLocalError: cannot access local variable 'idxDict' where it is not associated with a value
/
Hello, I ran FeGenie successfully (thanks)
I was wondering the meaning of such error message (in bold) at the end on the job. May you tell
I ran the following command./FeGenie.py -bin_dir ./FeGenie_bins -bin_ext fasta -t 4 -out output_fegenie5
checking arguments
.
.
.
All required arguments provided!
Finding ORFs for MTL4.1-CDSs.fasta
starting main pipeline...
.
Looking for following iron-related functional category: iron_storage
analyzing MTL4.1-CDSs.fasta: 100%
.
.........
.........
.
Looking for following iron-related functional category: magnetosome_formation
analyzing MTL4.1-CDSs.fasta: 100%
Consolidating summary files into one master summary file
Identifying genomic proximities and putative operons
Traceback (most recent call last):
File "./FeGenie.py", line 2170, in
main()
File "./FeGenie.py", line 843, in main
CoordDict[i][contig].append(int(numOrf))
ValueError: invalid literal for int() with base 10: '130050|ID:75309160|bioA'
Hi,
I have several hits in the geneSummary.csv file witn EMPTY in the protein sequence column. What does that mean? I provided ORFs to analyse but am unsure now if I can use the results.
iron_oxidation,protein-sequences.faa,713031,sulfocyanin,48.8,45.4,3638,0,0,0,MSVRTSTSVALSVGLGLSLVGGNALTPASAASSRYISYNAHNHTAKIILIGALNNSNQGMNFDGYANGKAIFTVPLGTKVTVAYSDGASKPHSAEIAPWSASIPAAAVTPAFKGAASADYANGSEKGDPISVFTFTASKAGKYRIMCGVTAHAILGMWDVLQVSKSAKVATLN
iron_oxidation,protein-sequences.faa,207671,Cyc2_repCluster2,298.9,27.8,3639,1,0,0,EMPTY
iron_oxidation,protein-sequences.faa,438947,Cyc2_repCluster2,333.9,27.8,3640,1,0,0,EMPTY
Thank you!!
Hi,thanks for your good jobs. I want to identify the iron-sulfur proteins of MAGs. I see that you will update this function in the near future. When will it be updated? is there a tool for this? thank you!
Hi, I'm interested in identifying iron related genes within a metatranscriptomics dataset. I have attempted providing a fasta with gene (nucleotide) or protein sequences as bellow, but I hit an error that says hmm not found. Please see below.
Thanks! Ana~
All required arguments provided!
HMM
AsbE_petrobactin_synthesis-rep
Cyc1
Cyc2_repCluster1
.
.
.
GACE_1846
GACE_1847
nramp
zip
starting main pipeline...
2
Looking for following iron-related functional category: iron_storage
sh: 1: hmmsearch: not foundta: 33%
rm: cannot remove 'TEST_out/test_proteins.fasta-HMM/Transferrin_TbpB_binding_protein_Haemophilus_influenzae_P44971.hmm.txt': No such file or directory
FeGenie cannot find the correct hmmsearch output files. If you provided gene or ORF-call sequences, please be sure to specify this in the command using the '--orfs' flag
Traceback (most recent call last):
File "/home/.../FeGenie/FeGenie.py", line 3083, in
main()
File "/home/.../FeGenie/FeGenie.py", line 821, in main
for line in hmmout:
UnboundLocalError: local variable 'hmmout' referenced before assignment
Running the quick-start fasta files directly following a conda install gives the following error:
FeGenie.py -bin_dir test_pao1/ -bin_ext txt -t 1
checking arguments
.
.
.
All required arguments provided!
Finding ORFs for Pseudomonas_aeruginosa_PA01.txt
starting main pipeline...
.
Looking for following iron-related functional category: iron_aquisition-heme_oxygenase
analyzing Pseudomonas_aeruginosa_PA01.txt: 100%
.
Looking for following iron-related functional category: iron_aquisition-heme_transport
analyzing Pseudomonas_aeruginosa_PA01.txt: 100%
.
Looking for following iron-related functional category: iron_aquisition-iron_transport
analyzing Pseudomonas_aeruginosa_PA01.txt: 100%
.
Looking for following iron-related functional category: iron_aquisition-siderophore_synthesis
analyzing Pseudomonas_aeruginosa_PA01.txt: 100%
.
Looking for following iron-related functional category: iron_aquisition-siderophore_transport
analyzing Pseudomonas_aeruginosa_PA01.txt: 100%
.
Looking for following iron-related functional category: iron_gene_regulation
analyzing Pseudomonas_aeruginosa_PA01.txt: 100%
.
Looking for following iron-related functional category: iron_oxidation
analyzing Pseudomonas_aeruginosa_PA01.txt: 100%
.
Looking for following iron-related functional category: iron_reduction
analyzing Pseudomonas_aeruginosa_PA01.txt: 100%
.
Looking for following iron-related functional category: iron_storage
analyzing Pseudomonas_aeruginosa_PA01.txt: 100%
.
Looking for following iron-related functional category: magnetosome_formation
analyzing Pseudomonas_aeruginosa_PA01.txt: 100%
Consolidating summary files into one master summary file
Identifying genomic proximities and putative operons
Clustering ORFs...
.
Looking for Thermincola S-layer cytochromes and Geobacter-related porin-cytochrome operons
Pre-processing of final outout file
Counting heme-binding motifs
Final processing of output
Traceback (most recent call last):
File "/home/snip/FeGenie/FeGenie.py", line 2167, in <module>
main()
File "/home/snip/FeGenie/FeGenie.py", line 1414, in main
memoryDict[dataset][orf]["seq"] = ls[10]
IndexError: list index out of range
Platform: Debian 10
Python 3.7.6
Hi,
Thanks for the pipeline. It made my work easy. I am interested in identifying iron siderophore synthesis in my metagenomic samples. When I run the fegenie pipeline without --all_results the output heatmap.csv shows iron_aquisition-siderophore_synthesis as 0,0,0,0,0.
if I include --all_results tag in my command it shows as iron_aquisition-siderophore_synthesis,2592,2488,5138,5511,1979.
It is very strange to get 0 counts considering my samples are from a natural environment and consist of a diverse bacterial community.
What's the difference between the above two commands (with and without --all_results tag)?
The help section says --all_results: report all results, regardless of clustering patterns and operon structure
I didn't clearly understand what does that actually means.
Does it have anything to do with search accuracy?
Hi. I'm trying to install FeGenie with conda (v4.8.3) but when I run the first command line for the conda install I get the following error messages:
Reading package lists... Done
Building dependency tree
Reading state information... Done
E: Unable to locate package clone
E: Unable to locate package https://github.com/Arkadiy-Garber
E: Couldn't find any package by glob 'https://github.com/Arkadiy-Garber'
E: Couldn't find any package by regex 'https://github.com/Arkadiy-Garber'
I'm running Ubuntu 18.04.4 in case that helps.
Hi Arkadiy,
Just a quick question -- are the outputs in the geneSummary files showing all encoded HMM hits and their ORFs or are these hits that are both encoded and have coverage in the supplied BAMs? Thanks!!
Best,
Joy
Hi,
Thank you for your tool. We would like to ask where the data for protein DmkA was obtained from. We didn't find the protein sequence of DmkA in NR database. And the proteins annotated as DmkA using FeGenie was re-annotated using NR database and the result was 1,4-dihydroxy-2-naphthoate octaprenyltransferase, prenyltransferase, etc.
Thank you!
Hey ! Thank for your tool :)
I am looking for gene involved in iron oxydation in genomes and MAGs. When Cyc2 is found there are not proteins sequences associated. It is the only gene for which it is noted as "empty" in protein_sequence column in geneSummary outputs. Is not normal ? if not, why do I always get this result ?
Thanks,
Eva
Hi there, and thank you for this tool! I am trying to run some metagenomes through FeGenie and have been successful so far, but I am trying to understand the numbers in FeGenie-heatmap-data.csv a little better. From what I understand, the default is gene counts divided by the predicted number of ORFs for each metagenome, multiplied by 1000. However, I have tried inputting '-inflation 100' (to get a percentage) and '-inflation 1' (to see the raw number), but the output is exactly the same as the default, making me question if I have done something wrong. All of my input files are .fasta, and I am also using the flag '--meta', if that is important. Most of the heatmap numbers are either 0 or in the low hundreds, ranging from 1 to 424.
Any insight would be very helpful. Thanks!
Hi, I just tested out FeGenie and it seems to be a very convenient tool to hunt for iron-metabolism-genes; many thanks for bringing it out for us in the microbiology community! I was wondering if you could shed more light on what those numbers actually represent in the output file FeGenie-heatmap-data.csv. I'm assuming "normalized abundance of genes per functional category" in each of the genomes being tested?
According to the README, the hmms/iron directory can be found within FeGenie's main repository. I was assuming that you meant the github repository, but I can't find it there. Can you point me in the right direction as to where to find this?
Hello Arkadiy,
--norm, if include this flag, the gene counts for each iron gene category to be normalized to the number of predicted ORFs in each genome or metagenome. Without normalization, FeGenie will create a heatmap-compatible CSV output with raw gene counts. With normalization, FeGenie will create a heatmap-compatible with 'normalized gene abundances'.
I compared the normalization and non-normalization data and found the normalized gene abundances if include this flag is 1000 times bigger than the percentage of “raw gene counts/predicted ORFs numbers” I calculated by myself. Is that correct? Thank you.
Error in hclust(d = dist(x = fegenie.scaled)) :
NA/NaN/Inf in foreign function call (arg 10)
Calls: as.dendrogram -> hclust
Execution halted
Hi Arkadiy, thanks very much for developing this great tool :)
I'm trying to implement FeGenie, however, near the end of the analysis this error is raised:
`Pre-processing of final outout file
Counting heme-binding motifs
mv: cannot move '../results/fegenie/FeGenie-summary-blasthits.csv/FeGenie-summary-blasthits.csv' to '../results/fegenie/FeGenie-summary-blasthits.csv/FeGenie-summary.csv': Permission denied
Final processing of output
Traceback (most recent call last):
File "/home/fooa/miniconda3/envs/fegenie/bin/FeGenie.py", line 3020, in
main()
File "/home/fooa/miniconda3/envs/fegenie/bin/FeGenie.py", line 1641, in main
infile = open(outDirectory + "/FeGenie-summary.csv")
FileNotFoundError: [Errno 2] No such file or directory: '../results/fegenie/FeGenie-summary-blasthits.csv/FeGenie-summary.csv'`
I still get the cluster.csv file but the summary.csv file is empty. I have a feeling this might be due to the server I'm using at my uni, but wondering if this is a problem you've encountered before or might have a solution for?
Thanks
Aidan
Hey! I do have another question!
After annotating my MAGs, I saw that FeGenie didn't find any transport-related clusters in any of my MAGs, which wouldn't make sense biologically (I have, among others, several cyanobacterial MAGs, and they must get their iron somewhere, right?). If I use the --all_results flag, I get some transport genes, but I'm not sure I should use them, since you mention in a different thread that this flag can create false-positives.
I imagine something goes wrong during the clustering step? I looked into one MAG specifically. According to the output produced by --all_results, it has the three EfeUOB genes, all next to each other, but they don't show up when I run the same MAG in strict mode. Are the other genes that should be present for the cluster to be complete?
Sorry for the basic question, I'm very new to the iron metabolism world :)
I can send you the MAG I looked into, or the output files, if needed.
Thanks!
Dear Arkadiy Garber,
I just watched your FeGenie lesson and tutorial in the BVCN and, first of all, I wanted to thank you for the nice tool (and teaching materials) you & your colleagues released!
I could easily install FeGenie in our server and now I am excited to give it a try.
I have a set of bins generated with several binning tools + das tool from 4 co-assembled metagenomes, so I have 4 bam files to which all bins relate.
My question is: would -bams work if a provide a tab-delimited file with all bin names in the first column and then, in the second column, a pathway to the folder containing the 4 bam files without specifying each bam file name?
From the tutorial I get this function is possible by having each bam file in a different column, right? In the near future I will get 12 co-assembled metagenomes and it would be easier to just point to a folder with all bam files :D
If this function is not supported yet, here is my suggestion :)
Thanks and best regards,
Paula Dalcin Martins
Dear Arkadiy,
I am getting the error below - could you help me understand what is going on and how to fix this?
Consolidating summary files into one master summary file
Identifying genomic proximities and putative operons
Traceback (most recent call last):
File "/proj/pdmartins/FeGenie/FeGenie.py", line 2170, in <module>
main()
File "/proj/pdmartins/FeGenie/FeGenie.py", line 843, in main
CoordDict[i][contig].append(int(numOrf))
ValueError: invalid literal for int() with base 10: 'protein'
I do get files such as FinalSummary.csv and .csv files for each category (i.e. iron_reduction-summary.csv, etc).
The commands I am running:
source activate fegenie
FeGenie.py -bin_dir /proj/pdmartins/2020_iron_reactor_analyses/analyses_2/ -bin_ext faa -t 16 -out /proj/pdmartins/2020_iron_reactor_analyses/analyses_2/fegenie --orfs --meta
The input file is a prodigal-generated and prokka-annotated fasta amino acid file.
Example of sequence in this file:
unbinned_NHGMMMNG_144129_2E-6E-farnesyl_diphosphate_synthase
MPDRITAGVDAVLDELLSERRLPDGLRGMMAYHLGWVDEDLRALPVRQRSKYGGKKMRAV
LCALACEAAGGDLETAFPAAAAVELVQNFSLVHDDIEDGDRERRHRPTVWVRWGVPQAIN
TGSAMQALVNAAVLRTPAPAETVLDVLRALTAAMVEMTEGQHLDIAFQDRTDVSVAEYED
MASRKTGALMEAAAYTGARLAMSDNRRLAAWRQFGRAFGQAFQARDDLLGVTGVPSVTGK
PVGNDIRARKKALPLLHALAHATPGDRVLLGRAFSNQAVSDEDVGRVTEVMERSGALDAT
RESVERATRSALEAFEATGALGPAADQIREMVSRAVGREQ
Here is the full slurm output file:
slurm-835700.txt
Thank you very much for your help!
Paula
Hi there,
Just installed FeGenie (again congrats on the tool!) via conda and tried running it on my bins as such: FeGenie.py -bin_dir Gal_FeGenie/ -bin_ext fa -t 16 -out Gal_FeGenie_out
.
Although in the end of the pipeline I do get the expected output files, as it runs I keep getting errors for all phases of the it past finding ORFs as such:
checking arguments
.
.
.
All required arguments provided!
Finding ORFs for Bin_24_1_1-contigs.fa
Finding ORFs for Bin_24_5_1-contigs.fa
Finding ORFs for Bin_28_2-contigs.fa
Finding ORFs for Bin_50_3-contigs.fa
Finding ORFs for Bin_56_3-contigs.fa
Finding ORFs for Bin_56_4-contigs.fa
Finding ORFs for Bin_56_6-contigs.fa
Finding ORFs for Bin_56_7_1-contigs.fa
Finding ORFs for Bin_56_8-contigs.fa
starting main pipeline...
.
Looking for following iron-related functional category: iron_aquisition-heme_oxygenase
analyzing Bin_24_1_1-contigs.fa: 100%
rm: cannot remove ‘Gal_FeGenie///Bin_24_1_1-contigs.fa-HMM’: Directory not empty
analyzing Bin_24_5_1-contigs.fa: 100%
rm: cannot remove ‘Gal_FeGenie///Bin_24_5_1-contigs.fa-HMM’: Directory not empty
analyzing Bin_28_2-contigs.fa: 100%
rm: cannot remove ‘Gal_FeGenie///Bin_28_2-contigs.fa-HMM’: Directory not empty
analyzing Bin_50_3-contigs.fa: 100%
rm: cannot remove ‘Gal_FeGenie///Bin_50_3-contigs.fa-HMM’: Directory not empty
analyzing Bin_56_3-contigs.fa: 100%
rm: cannot remove ‘Gal_FeGenie///Bin_56_3-contigs.fa-HMM’: Directory not empty
analyzing Bin_56_4-contigs.fa: 100%
rm: cannot remove ‘Gal_FeGenie///Bin_56_4-contigs.fa-HMM’: Directory not empty
analyzing Bin_56_6-contigs.fa: 100%
rm: cannot remove ‘Gal_FeGenie///Bin_56_6-contigs.fa-HMM’: Directory not empty
analyzing Bin_56_7_1-contigs.fa: 100%
rm: cannot remove ‘Gal_FeGenie///Bin_56_7_1-contigs.fa-HMM’: Directory not empty
analyzing Bin_56_8-contigs.fa: 100%
rm: cannot remove ‘Gal_FeGenie///Bin_56_8-contigs.fa-HMM’: Directory not empty
Any ideas? Can't be sure if the pipeline ran that well with all this...
Thanks in advance :)
Hi Arkadiy,
Thank you for all that you do! I am noticing that FeGenie renumbers/names contigs within individual MAGs' depth files. For those of us who cross-reference output from different tools, like anvi'o, would it be possible to retain the original contig names? For example, if I'm looking at output about c_000000001 from anvi'o and I use the same input fastas for FeGenie analysis, it would be really great to have that same c_000000001 contig be tied to the same MAG that anvi'o analyzed. Does this make sense? I double-checked that the input MAGs fastas for FeGenie were those made by the SUMMARIZE program within anvi'o.
Thank you!!
Best,
Joy
Hello, congratulations for your work.
I had an error while executing the last command on "Easy Installation (if you don't have Conda)" insctructions:
./FeGenie.py -h
, which throws to me:
Traceback (most recent call last): File "./FeGenie.py", line 459, in main test = open(bits) FileNotFoundError: [Errno 2] No such file or directory: '/HMM-bitcutoffs.txt' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "./FeGenie.py", line 2170, in <module> main() File "./FeGenie.py", line 467, in main location = allButTheLast(location, "/") UnboundLocalError: local variable 'location' referenced before assignment
With a little lookup, I replaced line #462 of FeGenie.py file from:
os.system("which FeGenie.py > mainDir.txt")
, to
os.system("which ./FeGenie.py > mainDir.txt")
, which solved the problem for me. Just in case anyone has the same error.
Hello, I've used your tool and it works great! I was wondering if you could add an option to output a nucleotide fasta file, similar to the proteins.faa file created during the Prodigal ORF searching. I would like to do some metatranscriptome mappings to the hits, and so this option would be quite useful. Thanks!
Hello,
I just downloaded your script (nonconda way) and hoped to use it to search for Fe associated genes in my metagenome. However, I ran into an error when I typed ./FeGenie.py -h
. This same error also occurred when I tried to run the program following your instruction ./FeGenie.py -hmm_lib hmms/iron -bin_dir /directory/of/bins/ -bin_ext fasta -t 16 -out output_fegenie
:
Traceback (most recent call last):
File "./FeGenie.py", line 506, in main
test = open(bits)
FileNotFoundError: [Errno 2] No such file or directory: '/HMM-bitcutoffs.txt'During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "./FeGenie.py", line 3006, in
main()
File "./FeGenie.py", line 514, in main
location = allButTheLast(location, "/")
UnboundLocalError: local variable 'location' referenced before assignment
Thanks!
Clare
Hi there,
I am aiming to make plots with coverage data for several bam files. I get .depth files and .csv files (as well as ORF_calls and HMM_results. However, there are no plots. Here is my input and the error:
FeGenie.py -bin_dir /home/lloydlab/klds1515/SVA08.16_Metagenomes/071817KLmetagenome-45705703/Bins_from_Derep_Dastool/Fasta_files/Indiv_Anvio_fastas -bin_ext fasta -out /klds1515/SVA08.16_Metagenomes/071817KLmetagenome-45705703/Bins_from_Derep_Dastool/Fegenie/Output/Plots -t 16 -bams Bam_file_for_fegenie.txt --makeplots --norm
Thread 0 finished: VKAB123_transcripts_MAGs_no_ribo.bam with 15432 reads and 8719 readsWellMapped
Creating depth matrix file: /klds1515/SVA08.16_Metagenomes/071817KLmetagenome-45705703/Bins_from_Derep_Dastool/Fegenie/Output/Plots/Woeseia_stnF.depth
Closing most bam files
Closing last bam file
Finished
processing... Woeseia_stnF
Traceback (most recent call last):
File "/home/lloydlab/anaconda3/envs/fegenie/bin/FeGenie.py", line 3020, in
main()
File "/home/lloydlab/anaconda3/envs/fegenie/bin/FeGenie.py", line 2586, in main
Dict[cell][process].append(float(depthDict[cell][contig]))
ValueError: could not convert string to float: 'EMPTY'
Thank you!
Hello Arkadiy,
I used FeGenie and I think it works very well and is simple to use. I am not sure where I can find more information about some of the HMM model calls. I checked the publication text and supplementary files, but can't find what the names DFE_0465 and DFE_0448 refer to.
This came up because I'm trying to reconcile some results between different annotation software.
I've attached the few instances where KEGG and FeGenie annotations disagree.
FeGenieCollisions.txt
I was able to find closely related sequences in NCBI refseq_protein using BLASTp, which i summarize below.
Any help on understanding how to reconcile these annotations and on where to find more information for the HMMs included in the software would be great.
Thanks,
Keith
Hi,
Thanks for the tool. Since I found that the annotations obtained from FeGenie installed by conda were few. I test the FeGenie installed by conda and the FeGenie installed manually. The results of these two approaches are very different. Why?
Thank you!
Hi there,
I've run into a new problem with running FeGenie. Previously I've run the script with no issue however now I receive the following error everytime I try to run
Consolidating summary files into one master summary file Identifying genomic proximities and putative operons Clustering ORFs... Looking for Thermincola S-layer cytochromes and Geobacter-related porin-cytochrome operons Pre-processing of final outout file rm: cannot remove 'fegenie_out4/GeoThermin.csv': Text file busy rm: cannot remove 'fegenie_out4/magnetosome_formation-summary.csv': Text file busy rm: cannot remove 'fegenie_out4/FinalSummary-dereplicated-clustered-blast.csv': Text file busy rm: cannot remove 'fegenie_out4/Afreen_L.plantarum202195_GCA_010586945.1_ASM1058694v1_genomic.fna-thermincola.blast': Text file busy Counting heme-binding motifs rm: cannot remove 'fegenie_out4/FeGenie-summary.csv': Text file busy mv: cannot move 'fegenie_out4/FeGenie-summary-blasthits.csv' to 'fegenie_out4/FeGenie-summary.csv': Operation not permitted Final processing of output Traceback (most recent call last): File "/home/qiime/FeGenie/FeGenie.py", line 2719, in <module> main() File "/home/qiime/FeGenie/FeGenie.py", line 1596, in main if ls[6] != "cluster": IndexError: list index out of range
I see that updates were made to the FeGenie.py script 27 days ago I'm assuming this may be a an error do to the addition of some categories, however I'm not sure how to troubleshoot it. Any help you can offer would be appreciated.
Thanks
Hi,
I've encountered an error during running FeGenie installed without conda.
The error message is as below.
#######
Traceback (most recent call last):
File "./FeGenie.py", line 458, in main
test = open(bits)
FileNotFoundError: [Errno 2] No such file or directory: '/HMM-bitcutoffs.txt'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "./FeGenie.py", line 1984, in
main()
File "./FeGenie.py", line 466, in main
location = allButTheLast(location, "/")
#######
This error can be solved by manually setting a path to "iron_hmms".
$ export iron_hmms=/path_to/FeGenie/hmms/iron/
Do I have to set the hmm path for FeGenie installed without conda?
Thanks.
Hi,
I have been playing with FeGenie on a small metagenome and successfully generated summary files of absolute #s of genes, normalized gene#s and gene coverage using a bam file. However to get each of these I had to rerun the whole thing each time from my fasta files.
For a v. large metagenome the gene finding and hmm searching parts are a time consuming step. Is there way to generate the multiple flavors of the summary output in the same run? Or to be able to direct FeGenie to use the ORF_calls and HMM_results folders generated by the previous run on the same fasta files?
Thanks
Gabrielle
Hi Arkadiy,
I am trying to run FeGenie with various bam file aligning to 2 different genomes. The command I am using is:
FeGenie.py -bin_dir final_megahit_co_assemblies/ -bin_ext fa -out fegenie_total_A_C_output -t 10 -bams WL_fegenie_bam_map_test.tsv --meta --makeplots
Except with the absolute paths, the thing is I keep getting the following as a output,
I have removed and reinstalled FeGenie and reconfigured my environment and etc but I can't figure out whats wrong.
here is the error output
...
Looking for Thermincola S-layer cytochromes and Geobacter-related porin-cytochrome operons
Pre-processing of final outout file
Counting heme-binding motifs
Final processing of output
Writen summary to file: fegenie_total_A_C_output/FeGenie-geneSummary-clusters.csv for visual inspection
Writen summary to file: fegenie_total_A_C_output/FeGenie-geneSummary.csv for downstream parsing and analyses
Writing heatmap-formatted output file: fegenie_total_A_C_output/FeGenie-heatmap-data.csv
processing... final_megahit_co_assemblies/WL_A_metaG_co_assemblies.fa
Output depth matrix to fegenie_total_A_C_output/final_megahit_co_assemblies/WL_A_metaG_co_assemblies.fa.depth
jgi_summarize_bam_contig_depths 2.15 (Bioconda) 2020-07-03T11:59:07
Output matrix to fegenie_total_A_C_output/final_megahit_co_assemblies/WL_A_metaG_co_assemblies.fa.depth
0: Opening bam: /Users/oliviaannerumble/bioinformatics/Waterlogging_project/Metatranscriptomic_WL_files/metaT_DRY_A/RQCF_DRY_A_files/Dry_A_bam_files/sorted_bam_files/WD-19_S10_L001_A_paired_mapped_sorted.bam
0: Opening bam: /Users/oliviaannerumble/bioinformatics/Waterlogging_project/Metatranscriptomic_WL_files/metaT_DRY_A/RQCF_DRY_A_files/Dry_A_bam_files/sorted_bam_files/WD-19_S10_L002_A_paired_mapped_sorted.bam
Processing bam files
WARNING: your aligner reports an incorrect NM field. You should run samtools calmd! nm < ins + del: cmatch=0 nm=33 ( insert=0 + del=44 + mismatch=33 == 77) D00472:89:HLFHMBCXX:1:1103:11614:22137 1:N:0:GAGATTCC+TAATCTTA
Thread 0 finished: WD-19_S10_L001_A_paired_mapped_sorted.bam with 1140574 reads and 117839 readsWellMapped
Thread 0 finished: WD-19_S10_L002_A_paired_mapped_sorted.bam with 1168778 reads and 120993 readsWellMapped
Creating depth matrix file: fegenie_total_A_C_output/final_megahit_co_assemblies/WL_A_metaG_co_assemblies.fa.depth
Closing most bam files
Closing last bam file
Finished
processing... final_megahit_co_assemblies/WL_A_metaG_co_assemblies.fa
Traceback (most recent call last):
File "/Users/oliviaannerumble/miniconda3/envs/fegenie/bin/FeGenie.py", line 2636, in main
depth = open("%s/contigDepths/%s.depth" % (args.out, cell))
FileNotFoundError: [Errno 2] No such file or directory: 'fegenie_total_A_C_output/contigDepths/final_megahit_co_assemblies/WL_A_metaG_co_assemblies.fa.depth'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/Users/oliviaannerumble/miniconda3/envs/fegenie/bin/FeGenie.py", line 3139, in
main()
File "/Users/oliviaannerumble/miniconda3/envs/fegenie/bin/FeGenie.py", line 2652, in main
depth = open("%s/%s.depth" % (outDirectory, cell))
FileNotFoundError: [Errno 2] No such file or directory: 'fegenie_total_A_C_output/final_megahit_co_assemblies/WL_A_metaG_co_assemblies.fa.depth'
Hi,
I would like to create conda package for FeGenie.
In order to do this, having a license file that allows distribution in the repo is required, and having official tagged release is preferable.
Thus, I was wondering if you can add license file, as well as tag a release of FeGenie.
Thank you,
Natasha
Hi,
Thank you for your tool. I am currently using FeGenie on contigs and have tried to use it on ORFs as well. My results are quite different and those for ORFs are incoherent. I have the impression that in the current version, there is a problem with the use of ORFs. I just saw that you have put in your Upcoming Updates "the ability to accept previously annotated genomes and gene calls". Is this coming soon?
Thank you!
After running FeGenie with a reference database (nr.dmnd) I get the error shown below.
As an output I get the complete FeGenie-summary.csv and an empty file FeGenie-summary-altered.csv.
I think I can work with the complete summary file, but still wanted to report the bug.
Command
FeGenie.py -bin_dir . -bin_ext fsa -out ../output_fegenie_seqs/ --meta -ref /share/references/library/nr -t 10
Error:
Performing Diamond BLASTx search of putative iron genes against reference database
Counting heme-binding motifs
Final processing of output
Traceback (most recent call last):
File "/XXX/XXX/miniconda3/envs/fegenie/bin/FeGenie.py", line 3020, in <module>
main()
File "/XXX/XXX/miniconda3/envs/fegenie/bin/FeGenie.py", line 2355, in main
ls[0] + "," + ls[1] + "," + str(idxDict[ls[2]]) + "," + ls[3] + "," + ls[4] + "," + ls[
UnboundLocalError: local variable 'idxDict' referenced before assignment
Hi Arkadiy
Thanks for your work developing FeGenie. I installed it on a MacBookPro with Conda and the basic commands are working on my dataset, but when I tried to use the -bam option I got the following error.
processing... GeoMICS_all_contigs.fa
sh: jgi_summarize_bam_contig_depths: command not found
processing... GeoMICS_all_contigs.fa
Traceback (most recent call last):
File "/Users/gabrielle/Desktop/GenomeFun/FeGenie/FeGenie.py", line 2293, in main
depth = open("%s/contigDepths/%s.depth" % (args.out, cell))
FileNotFoundError: [Errno 2] No such file or directory: 'geomics_all_bam_test_out/contigDepths/GeoMICS_all_contigs.fa.depth'
I recognized this script name from my prior efforts at binning this metagenome, so I just installed metabat2 in the same Conda environment and now the bam option appears to work. I am sure this is not the preferred solution, posting here in case it helps someone else or to get a better fix.
Please help. Running the command (bash setup_noconda.sh) returns an error as given below:
test_dataset/
test_dataset/Aggregatibacter_actinomycetemcomitans.txt
test_dataset/Mariprofundus_ferrooxydans_PV-1.txt
test_dataset/Streptococcus_mutans.txt
test_dataset/Pseudomonas_aeruginosa_PA01.txt
test_dataset/Porphyromonas_gingivalis.txt
test_dataset/Rhodopseudomonas_palustris_TIE-1.txt
test_dataset/Geobacter_bemidjiensis.txt
test_dataset/Shewanella_oneidensis_MR-1.txt
test_dataset/Magnetospirillum_magneticum_AMB-1.txt
test_dataset/Rhodopseudomonas_palustris_TIE-1.txt-proteins.faa
test_dataset/Acidithiobacillus_ferrooxidans.txt
Installing package into ‘/usr/lib64/R/library’
(as ‘lib’ is unspecified)
Warning in install.packages("grid", repos = "http://cran.us.r-project.org") :
'lib = "/usr/lib64/R/library"' is not writable
Error in install.packages("grid", repos = "http://cran.us.r-project.org") :
unable to install packages
Execution halted
Error: unexpected end of input
Execution halted
setup_noconda.sh: line 8: syntax error near unexpected token (' setup_noconda.sh: line 8:
Rscript -e 'install.packages("ggplot2", repos = "http://cran.us.r-project.org")''
How can I overcome this issue?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.