Hi,
I am running ODM with GPU support on a Nvidia DGX station, however it will only use about 800MB of the GPU even though the DGX has 4 times 40GB of GPU.
I run the following docker command with the --gpus all flag as explained on the github repository (GitHub - OpenDroneMap/ODM: A command line toolkit to generate maps, point clouds, 3D models and DEMs from drone, balloon or kite images. 📷):
docker run -ti --rm -v c:/Users/youruser/datasets:/datasets --gpus all opendronemap/odm:gpu --project-path /datasets project --orthophoto-resolution 0.5 --fast-orthophoto --skip-3dmodel
when I open an interactive session within the docker container, the nvidia-smi command is available and there is only one process using the GPU, which is python3.
The NVIDIA-SMI, Driver Version and CUDA Version is the following:
±----------------------------------------------------------------------------+
| NVIDIA-SMI 525.147.05 Driver Version: 525.147.05 CUDA Version: 12.0 |
|-------------------------------±---------------------±---------------------+
I have the newest docker version installed and the cuda release 11.4:
$nvcc --version
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2021 NVIDIA Corporation
Built on Mon_Oct_11_21:27:02_PDT_2021
Cuda compilation tools, release 11.4, V11.4.152
Build cuda_11.4.r11.4/compiler.30521435_0
I downloaded the git repository and adjusted the gpu.Dockerfile, so it builds from:
FROM nvidia/cuda:11.4.3-devel-ubuntu20.04 AS builder
instead of:
FROM nvidia/cuda:11.2.2-devel-ubuntu20.04 AS builder
Building this does not have an effect though. It still uses only about 800MB of the GPU.
I tried the same thing on a different ubuntu server, 16GB of GPU and I get the same results.
I am trying to create an orthophoto with a high resolution, but using the GPU does not make it 2x faster than using only the CPU as mentioned on the github repository. It takes the same amount of time with or without using the GPU.
After reviewing the forum, it appears that many others are experiencing the same issue where the GPU is not being utilized. Unfortunately, I was unable to find a solution to this problem either on the forum or elsewhere.
Hope someone has a solution.