Comments (4)
Hi @michoug
Can you check which version of Tensorflow you have installed? Just run python -c "import tensorflow as tf;print(tf.__version__)"
.
Also, does your machine have a GPU? It seems that it is the GPU that is running out of memory. If that's the case, I can release a fix that forces the use of the CPU.
In the meantime, you can try to run the genomad nn-classification
command with a smaller batch size (for example, --batch-size 32
or --batch-size 16
). If this works, you can run the end-to-end
command to execute the remaining modules.
from genomad.
Hi @apcamargo
The tenserflow version is 2.11.0
and there is a GPU with 4 Gb of memory (NVIDIA T400 Gb).
When running with --batch-size 32
, I got the same error but with --batch-size 16
, I got another one.
Probably a driver issue on my part
[16:02:20] Executing genomad nn-classification.
[16:02:20] Creating the Genomad/336R_concoct_107_genomad/336R_concoct_107_nn_classification/336R_concoct_107_encoded_sequences directory.
[16:02:22] Encoded sequence data written to 336R_concoct_107_encoded_sequences.
⠹ Classifying sequences.2023-03-15 16:02:26.054685: E tensorflow/compiler/xla/stream_executor/cuda/cuda_dnn.cc:433] Could not create cudnn handle: CUDNN_STATUS_NOT_INITIALIZED
2023-03-15 16:02:26.054820: E tensorflow/compiler/xla/stream_executor/cuda/cuda_dnn.cc:442] Possibly insufficient driver version: 510.108.3
Traceback (most recent call last):
File "/home/river/miniconda3/envs/genomad/bin/genomad", line 10, in <module>
sys.exit(cli())
File "/home/river/miniconda3/envs/genomad/lib/python3.10/site-packages/click/core.py", line 1130, in __call__
return self.main(*args, **kwargs)
File "/home/river/miniconda3/envs/genomad/lib/python3.10/site-packages/rich_click/rich_group.py", line 21, in main
rv = super().main(*args, standalone_mode=False, **kwargs)
File "/home/river/miniconda3/envs/genomad/lib/python3.10/site-packages/click/core.py", line 1055, in main
rv = self.invoke(ctx)
File "/home/river/miniconda3/envs/genomad/lib/python3.10/site-packages/click/core.py", line 1657, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/home/river/miniconda3/envs/genomad/lib/python3.10/site-packages/click/core.py", line 1404, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/home/river/miniconda3/envs/genomad/lib/python3.10/site-packages/click/core.py", line 760, in invoke
return __callback(*args, **kwargs)
File "/home/river/miniconda3/envs/genomad/lib/python3.10/site-packages/genomad/cli.py", line 694, in nn_classification
genomad.nn_classification.main(
File "/home/river/miniconda3/envs/genomad/lib/python3.10/site-packages/genomad/modules/nn_classification.py", line 285, in main
contig_predictions = nn_model.predict(
File "/home/river/miniconda3/envs/genomad/lib/python3.10/site-packages/keras/utils/traceback_utils.py", line 70, in error_handler
raise e.with_traceback(filtered_tb) from None
File "/home/river/miniconda3/envs/genomad/lib/python3.10/site-packages/tensorflow/python/eager/execute.py", line 52, in quick_execute
tensors = pywrap_tfe.TFE_Py_Execute(ctx._handle, device_name, op_name,
tensorflow.python.framework.errors_impl.UnimplementedError: Graph execution error:
Detected at node 'model_1/model/conv1d/Conv1D' defined at (most recent call last):
File "/home/river/miniconda3/envs/genomad/bin/genomad", line 10, in <module>
sys.exit(cli())
File "/home/river/miniconda3/envs/genomad/lib/python3.10/site-packages/click/core.py", line 1130, in __call__
return self.main(*args, **kwargs)
File "/home/river/miniconda3/envs/genomad/lib/python3.10/site-packages/rich_click/rich_group.py", line 21, in main
rv = super().main(*args, standalone_mode=False, **kwargs)
File "/home/river/miniconda3/envs/genomad/lib/python3.10/site-packages/click/core.py", line 1055, in main
rv = self.invoke(ctx)
File "/home/river/miniconda3/envs/genomad/lib/python3.10/site-packages/click/core.py", line 1657, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/home/river/miniconda3/envs/genomad/lib/python3.10/site-packages/click/core.py", line 1404, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/home/river/miniconda3/envs/genomad/lib/python3.10/site-packages/click/core.py", line 760, in invoke
return __callback(*args, **kwargs)
File "/home/river/miniconda3/envs/genomad/lib/python3.10/site-packages/genomad/cli.py", line 694, in nn_classification
genomad.nn_classification.main(
File "/home/river/miniconda3/envs/genomad/lib/python3.10/site-packages/genomad/modules/nn_classification.py", line 285, in main
contig_predictions = nn_model.predict(
File "/home/river/miniconda3/envs/genomad/lib/python3.10/site-packages/keras/utils/traceback_utils.py", line 65, in error_handler
return fn(*args, **kwargs)
File "/home/river/miniconda3/envs/genomad/lib/python3.10/site-packages/keras/engine/training.py", line 2350, in predict
tmp_batch_outputs = self.predict_function(iterator)
File "/home/river/miniconda3/envs/genomad/lib/python3.10/site-packages/keras/engine/training.py", line 2137, in predict_function
return step_function(self, iterator)
File "/home/river/miniconda3/envs/genomad/lib/python3.10/site-packages/keras/engine/training.py", line 2123, in step_function
outputs = model.distribute_strategy.run(run_step, args=(data,))
File "/home/river/miniconda3/envs/genomad/lib/python3.10/site-packages/keras/engine/training.py", line 2111, in run_step
outputs = model.predict_step(data)
File "/home/river/miniconda3/envs/genomad/lib/python3.10/site-packages/keras/engine/training.py", line 2079, in predict_step
return self(x, training=False)
File "/home/river/miniconda3/envs/genomad/lib/python3.10/site-packages/keras/utils/traceback_utils.py", line 65, in error_handler
return fn(*args, **kwargs)
File "/home/river/miniconda3/envs/genomad/lib/python3.10/site-packages/keras/engine/training.py", line 561, in __call__
return super().__call__(*args, **kwargs)
File "/home/river/miniconda3/envs/genomad/lib/python3.10/site-packages/keras/utils/traceback_utils.py", line 65, in error_handler
return fn(*args, **kwargs)
File "/home/river/miniconda3/envs/genomad/lib/python3.10/site-packages/keras/engine/base_layer.py", line 1132, in __call__
outputs = call_fn(inputs, *args, **kwargs)
File "/home/river/miniconda3/envs/genomad/lib/python3.10/site-packages/keras/utils/traceback_utils.py", line 96, in error_handler
return fn(*args, **kwargs)
File "/home/river/miniconda3/envs/genomad/lib/python3.10/site-packages/keras/engine/functional.py", line 511, in call
return self._run_internal_graph(inputs, training=training, mask=mask)
File "/home/river/miniconda3/envs/genomad/lib/python3.10/site-packages/keras/engine/functional.py", line 668, in _run_internal_graph
outputs = node.layer(*args, **kwargs)
File "/home/river/miniconda3/envs/genomad/lib/python3.10/site-packages/keras/utils/traceback_utils.py", line 65, in error_handler
return fn(*args, **kwargs)
File "/home/river/miniconda3/envs/genomad/lib/python3.10/site-packages/keras/engine/training.py", line 561, in __call__
return super().__call__(*args, **kwargs)
File "/home/river/miniconda3/envs/genomad/lib/python3.10/site-packages/keras/utils/traceback_utils.py", line 65, in error_handler
return fn(*args, **kwargs)
File "/home/river/miniconda3/envs/genomad/lib/python3.10/site-packages/keras/engine/base_layer.py", line 1132, in __call__
outputs = call_fn(inputs, *args, **kwargs)
File "/home/river/miniconda3/envs/genomad/lib/python3.10/site-packages/keras/utils/traceback_utils.py", line 96, in error_handler
return fn(*args, **kwargs)
File "/home/river/miniconda3/envs/genomad/lib/python3.10/site-packages/keras/engine/functional.py", line 511, in call
return self._run_internal_graph(inputs, training=training, mask=mask)
File "/home/river/miniconda3/envs/genomad/lib/python3.10/site-packages/keras/engine/functional.py", line 668, in _run_internal_graph
outputs = node.layer(*args, **kwargs)
File "/home/river/miniconda3/envs/genomad/lib/python3.10/site-packages/keras/utils/traceback_utils.py", line 65, in error_handler
return fn(*args, **kwargs)
File "/home/river/miniconda3/envs/genomad/lib/python3.10/site-packages/keras/engine/base_layer.py", line 1132, in __call__
outputs = call_fn(inputs, *args, **kwargs)
File "/home/river/miniconda3/envs/genomad/lib/python3.10/site-packages/keras/utils/traceback_utils.py", line 96, in error_handler
return fn(*args, **kwargs)
File "/home/river/miniconda3/envs/genomad/lib/python3.10/site-packages/keras/layers/convolutional/base_conv.py", line 283, in call
outputs = self.convolution_op(inputs, self.kernel)
File "/home/river/miniconda3/envs/genomad/lib/python3.10/site-packages/keras/layers/convolutional/base_conv.py", line 255, in convolution_op
return tf.nn.convolution(
Node: 'model_1/model/conv1d/Conv1D'
DNN library is not found.
[[{{node model_1/model/conv1d/Conv1D}}]] [Op:__inference_predict_function_942]
from genomad.
Hi,
I resolved all the issues by installing specific version of cudatoolkit=11.2.2
and cudnn=8.1.0
Best
Greg
from genomad.
Thanks for the feedback, @michoug. I'll add instructions for running geNomad on a GPU on a future update.
This report was really useful!
from genomad.
Related Issues (20)
- Error downloading database HOT 2
- Inquiry on virus from MAG HOT 4
- [feature request] query database clustering HOT 1
- Whether measures have been taken by genomad to avoid identifying genomic islands as viruses? HOT 5
- AMR annotations on chromsome? HOT 1
- Errors when download and the same issue when running genomad -h HOT 3
- The virus identified by genomad weren't annotated as virus sequence by VIBRANT? HOT 3
- geNomad taxonomy about Baltimore classification HOT 1
- Error with geNomad v1.8.0, missing tensorflow.keras HOT 5
- mmseqs2 error HOT 3
- Different protein number from genomad and pyrodigal-gv HOT 2
- Small (reference) data for testing HOT 9
- Error while classifying sequences HOT 6
- Error mmseqs prefilter HOT 4
- genomad annotate fastq file is empty or contains multiple entries HOT 3
- plasmid classified as virus? HOT 7
- Optimization Request for Analyzing Large Number of MAGs with geNomad HOT 5
- Fewer viral contigs identified from genomad vs virsorter2 HOT 4
- The question about --disable-nn-classification HOT 1
- Provirus detection in genomad vs checkv HOT 6
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from genomad.