Coder Social home page Coder Social logo

predictf's People

Contributors

mdsufz avatar

Stargazers

avatar avatar avatar avatar

Watchers

avatar

Forkers

dna2rna nailouzhang

predictf's Issues

PATHing and diamond version conflict with supplied databases

Hello developers,

I am trying to run PredicTF and am running into some issues. The installation (on Ubuntu 16.04.6 LTS, 40 cores, 1.5TB RAM, local installation, no GPUs on this system(and hence i dont want to do any model training)) ran without any problems according your instructions but when trying to use your script predictf_in_genome.sh to use your model on my genomes, i ran into the following issues:

  1. The diamond version specified in this repository (installed via conda install -c bioconda diamond==0.9.24 ) is incompatible with the supplied databases used in the predictf_in_genome.sh wrapper script(Error: Database was built with a different version of Diamond and is incompatible). I checked that the specified version of diamond was used using the diamond version command. I can likely fix this by recompiling the features.fasta file in the same directory to the features.dmnd file (i assume from the same prefixes that this is the original .fasta file and not the deeparg.fasta file in the same folder).
  2. quite a few of the PATHs do not work for me, e.g., diamond was searched for in '/deeparg//bin/diamond' which is likely never the correct directory (fixed by removing the path specifications in the deeparg.py file so that it could pick the version itself), one of the model files is looked for in '/home/till/03bioinformaticstools/102predictTF/PredicTF/BacTFDB/BacTFDB/model/v2/metadata_LS.pkl' with a duplicated 'BacTFDB' folder in the PATH (the correct PATH would only have a single instance of BacTFDB/ ; i can modify this one as well)

The command i ran was:

sh predictf_in_genome.sh ~/03bioinformaticstools/102predictTF/PredicTF ../../genome/WB01no11_MG_WBTS_GAME_concat_Anaerolineae_51_11.genes.faa ../../genome/WB01no11_MG_WBTS_GAME_concat_Anaerolineae_51_11_out

and the console output i got was(i had run the same command previously, hence the mkdir warning; there seem to be additional calls to diamond also using the wrong PATH that i havent modified yet to the relative PATH; hence only the first db call of diamond works and the later one complains about the /deeparg//bin/diamond PATH):

mkdir: cannot create directory '../../genome/WB01no11_MG_WBTS_GAME_concat_Anaerolineae_51_11_out': File exists
path
/home/till/anaconda3/envs/deeparg_env/lib/python2.7/site-packages/theano/tensor/signal/downsample.py:6: UserWarning: downsample module has been moved to the theano.tensor.signal.pool module.
  "downsample module has been moved to the theano.tensor.signal.pool module.")
DIAMOND blastp alignment
diamond  blastp -q ../../genome/WB01no11_MG_WBTS_GAME_concat_Anaerolineae_51_11.genes.faa -d /home/till/03bioinformaticstools/102predictTF/PredicTF/BacTFDB//database/v2/features -k 1000 --id 30 --sensitive -e 1e-10 -a ../../genome/WB01no11_MG_WBTS_GAME_concat_Anaerolineae_51_11_out/file.out.align
diamond v0.9.24.125 | by Benjamin Buchfink <[email protected]>
Licensed under the GNU GPL <https://www.gnu.org/licenses/gpl.txt>
Check http://github.com/bbuchfink/diamond for updates.

#CPU threads: 40
Scoring parameters: (Matrix=BLOSUM62 Lambda=0.267 K=0.041 Penalties=11/1)
Temporary directory: ../../genome/WB01no11_MG_WBTS_GAME_concat_Anaerolineae_51_11_out
Opening the database...  [8.3e-05s]
Error: Database was built with a different version of Diamond and is incompatible.
parsing output file
sh: 1: /deeparg//bin/diamond: not found
Loading deep learning model ...
Traceback (most recent call last):
  File "/home/till/03bioinformaticstools/102predictTF/PredicTF/deeparg-largerepo/deepARG.py", line 157, in <module>
    '.mapping', iden, mdl, evalue, prob, minCoverage, pipeline, version)
  File "/home/till/03bioinformaticstools/102predictTF/PredicTF/deeparg-largerepo/predict/bin/deepARG.py", line 149, in process
    open(path+"BacTFDB/model/"+version_m+"/metadata"+version+".pkl"))
IOError: [Errno 2] No such file or directory: '/home/till/03bioinformaticstools/102predictTF/PredicTF/BacTFDB/BacTFDB/model/v2/metadata_LS.pkl'

If i remove the duplicated BacTFDB, i now get a cPickle error (using the same command; this one indicates to me that you may be using a different pickle version though cPickle was installed into the conda env and you thus at least indirectly install it with the installation commands):

mkdir: cannot create directory '../../genome/WB01no11_MG_WBTS_GAME_concat_Anaerolineae_51_11_out': File exists
path
/home/till/anaconda3/envs/deeparg_env/lib/python2.7/site-packages/theano/tensor/signal/downsample.py:6: UserWarning: downsample module has been moved to the theano.tensor.signal.pool module.
  "downsample module has been moved to the theano.tensor.signal.pool module.")
DIAMOND blastp alignment
diamond  blastp -q ../../genome/WB01no11_MG_WBTS_GAME_concat_Anaerolineae_51_11.genes.faa -d /home/till/03bioinformaticstools/102predictTF/PredicTF/BacTFDB//database/v2/features -k 1000 --id 30 --sensitive -e 1e-10 -a ../../genome/WB01no11_MG_WBTS_GAME_concat_Anaerolineae_51_11_out/file.out.align
diamond v0.9.24.125 | by Benjamin Buchfink <[email protected]>
Licensed under the GNU GPL <https://www.gnu.org/licenses/gpl.txt>
Check http://github.com/bbuchfink/diamond for updates.

#CPU threads: 40
Scoring parameters: (Matrix=BLOSUM62 Lambda=0.267 K=0.041 Penalties=11/1)
Temporary directory: ../../genome/WB01no11_MG_WBTS_GAME_concat_Anaerolineae_51_11_out
Opening the database...  [0.00017s]
Error: Database was built with a different version of Diamond and is incompatible.
parsing output file
sh: 1: /deeparg//bin/diamond: not found
Loading deep learning model ...
Traceback (most recent call last):
  File "/home/till/03bioinformaticstools/102predictTF/PredicTF/deeparg-largerepo/deepARG.py", line 157, in <module>
    '.mapping', iden, mdl, evalue, prob, minCoverage, pipeline, version)
  File "/home/till/03bioinformaticstools/102predictTF/PredicTF/deeparg-largerepo/predict/bin/deepARG.py", line 149, in process
    open(path+"/model/"+version_m+"/metadata"+version+".pkl"))
cPickle.UnpicklingError: invalid load key, 'v'.

Is there anything you can suggest to fix these errors?

Best regards,
Till

PS: Maybe adding a little note on how to limit core/thread usage to the README would be nice as users might not want to allocate all the 40 threads for this (i can modify the deeparg.py to that effect but it might be a nice info to share).

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.