medaka GPU about medaka

[cjw85]

The error cuda/cuda_dnn.cc:82] OOM when allocating tensor with shape[1536000000] indicates the GPU has run out of memory. The defaults in medaka are set such that it is runnable on GPUs with 12Gb for both training an inference.

Can you be sure that another process was not using the GPU concurrently, or that medaka was not accessing a GPU that it should not have been? Typically one would set the environment variable CUDA_VISIBLE_DEVICES=n to limit medaka/tensorflow to a single GPU, it will not run faster with multiple GPUs.

[MichelMoser]

Thanks for the suggestion. Indeed, guppy_basecall_server is occupying the GPUs. Should have stopped it first.