GPU test reports

guano · 30 December 2022 15:13

Hello everyone

Just reporting some tests for CPUxGPU

A few days ago I was at the beach (summer in Brazil!) and collected some images. I was using a machine with Ubuntu 22.10 and ODM 3.0

When I processed a dataset of 280 images, it took 2:30h (CPU only).

Since I wanted to try the GPU acceleration, I had to use a machine with Ubuntu 22.04 (22.10 is not officially supported by NVIDIA).

So I rolled back my machine to 22.04 and installed NIVIDIA drivers and docker-nvidia. Everything working fine:

+-----------------------------------------------------------------------------+
| NVIDIA-SMI 525.60.13    Driver Version: 525.60.13    CUDA Version: 12.0     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  NVIDIA GeForce ...  On   | 00000000:2E:00.0 Off |                  N/A |
| N/A   41C    P0    N/A /  N/A |      5MiB /  4096MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+
                                                                               
+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
|    0   N/A  N/A      3480      G   /usr/lib/xorg/Xorg                  4MiB |
+-----------------------------------------------------------------------------+

Ok, then I updated ODM/WebODM with GPU support and here I’d like to point that it would be really nice to have some better indication that we are using GPU. maybe show which docker image is being used? Maybe a flag?

Apart from that, I tested two datasets, one with 92 photos and one with 280 photos.

The one with 92 photos took 19 minutes without GPU and 11 minutes with GPU

The one with 280 images took 1 hour without GPU and only 22 minutes with GPU. Nice!

But I still don’t see why the CPU processing took 1 hour with ubuntu 22.04 and 2:30h with ubuntu 22.10… (and at this point I don’t have how to test it again)

It’s interesting that with CPU the reconstructed are is larger than with CPU

Still more tests to go, but I’m happy with the results so far!

I’ll leave the assets available in my Dropbox for a couple of months if someone is interested:

Happy 2023 to all!

Saijin_Naib · 30 December 2022 18:16

Great tests!

22.10 might have more kernel mitigations enabled for CPU Security flaws which can have pretty massive performance penalties.

As for GPU reconstruction area being smaller, it seems that GPU-based feature detection and matching, while many-fold faster, is a bit less robust.

I think of it like GPU-based video encoding engines. They are brutally fast, but they never match the compression ratio and perceptual quality of CPU-based compression, even after something like almost 20 years of common-place usage.