Run Darknet in Docker with Jetpack v5 about opendatacam

[vsaw]

Finally got it all working under Docker. Needs to clean everything up, and create a pull request, but the hard part is over :)

[vsaw]

At the moment it feels like a configuration issue with my device as I get output errors in nvidia-container-cli

$ nvidia-container-cli info
NVRM version:   (null)
CUDA version:   11.4

Device Index:   0
Device Minor:   0
Model:          Xavier
Brand:          (null)
GPU UUID:       (null)
Bus Location:   (null)
Architecture:   7.2

[vsaw]

Opened a thread in the NVIDIA Forums to see how to get GPU access in Docker working https://forums.developer.nvidia.com/t/nvidia-container-cli-does-not-show-nvrm-version-or-gpu-information/266204

[vsaw]

Going deeper into debugging I believe the issue is with the resinplayground/jetson-nano-cuda-cudnn-opencv:v0.2-slim image we use as a base for our Darknet image. It looks like this image does not detect the GPU.

I'm currently looking into different ways to dockerize Darknet with GPU access such as https://github.com/daisukekobayashi/darknet-docker/tree/master which uses a pre-built container by NVIDIA with CUDA and cuDNN (that is also much smaller then resinplayground!)

[vsaw]

daisukekobayashi only offers AMD64 builds, which we can not run on the Jetson devices, will therefore attempt to rebuild his image for ARM. If this works it could be a way to get ODC running on JetPack 5 and solve #608 at the same time!

This issue has been updated to show the new status of development.

[vsaw]

While I managed to build an ARM version of https://github.com/daisukekobayashi/darknet-docker/ I could not yet run Darknet in Docker. It always freezes with the following output

 CUDA-version: 11020 (11040), cuDNN: 8.1.1, CUDNN_HALF=1, GPU count: 1  
 CUDNN_HALF=1 
 OpenCV version: 4.2.0
 0 : compute_capability = 720, cudnn_half = 1, GPU: Xavier 
net.optimized_memory = 0 
mini_batch = 1, batch = 1, time_steps = 1, train = 0 
   layer   filters  size/strd(dil)      input                output
   0 Create CUDA-stream - 0 

The same error was already reported to the upstream repository AlexeyAB/darknet#8676

[vsaw]

Still no luck, opened a new issue in the official container runkit to ask for additional help NVIDIA/nvidia-container-toolkit#124

[vsaw]

I got it working with https://catalog.ngc.nvidia.com/orgs/nvidia/containers/l4t-jetpack to the point where I can run a JPEG through Darknet in Docker and it uses the GPU. But the resulting production image is 9.8GB large.

I checked how the base image is built and it includes a bunch of software we don't need for Darknet. I will check if I can build a custom base image with only CUDA, cuDNN and OpenCV to make the image slimmer.

[vsaw]

Here's the Dockerfile for https://catalog.ngc.nvidia.com/orgs/nvidia/containers/l4t-jetpack: https://gitlab.com/nvidia/container-images/l4t-jetpack/-/blob/master/Dockerfile.jetpack

The file looks straightforward so I will try to copy the important parts from there.

Here's the Docker Base https://catalog.ngc.nvidia.com/orgs/nvidia/containers/l4t-base for the container.

[vsaw]

Hmm, the docker base Image I used is not available for amd64 and therefore the Desktop version. I will only update the ARM builds (Jetson Nano and Xavier) for now.

[vsaw]

Released with https://github.com/opendatacam/darknet/releases/tag/v2.0.0-beta1