OpenDroneMap/ODM:gpu crashing

ChrisDAT · 25 March 2022 19:20

Hello,

i downloaded a docker image today with ODM and GPU support.
When i run it on a couple of images, i get the following exception:

[INFO]    running "/code/SuperBuild/install/bin/FPCFilter" --input "/code/opensfm/reconstruction.ply" --output "/code/odm_filterpoints/point_cloud.ply" --concurrency 32  --meank 16 --std 5.0 --boundary "/tmp/tmp_43tn96h.boundary.json"
 *** FPCFilter - v0.1 ***

?> Parameters:
        input = /code/opensfm/reconstruction.ply
        output = /code/odm_filterpoints/point_cloud.ply
        std = 5
        meanK = 16
        boundary = 140 polygon vertexes
        concurrency = 32
        verbose = no

 -> Setting num_threads to 32

 -> Cropping

 -> Done cropping in 0.4552s

 ?> Skipping sampling

 -> Statistical filtering

 ?> Done in 0.2s

 -> Writing output

 ?> Done in 0.03291s

 ?> Pipeline done in 0.6882s

[INFO]    Finished odm_filterpoints stage
[INFO]    Running odm_meshing stage
[INFO]    Writing ODM Mesh file in: /code/odm_meshing/odm_mesh.ply
[INFO]    running "/code/SuperBuild/install/bin/PoissonRecon" --in "/code/odm_filterpoints/point_cloud.ply" --out "/code/odm_meshing/odm_mesh.dirty.ply" --depth 10 --pointWeight 4.0 --samplesPerNode 1.0 --threads 31 --bType 2 --linearFit
[ERROR] Src/PoissonRecon.cpp (Line 1144)
        main
        Ply file does not contain normals
[WARNING] PoissonRecon failed with 15 threads, let's retry with 7...
[INFO]    running "/code/SuperBuild/install/bin/PoissonRecon" --in "/code/odm_filterpoints/point_cloud.ply" --out "/code/odm_meshing/odm_mesh.dirty.ply" --depth 10 --pointWeight 4.0 --samplesPerNode 1.0 --threads 15 --bType 2 --linearFit
[ERROR] Src/PoissonRecon.cpp (Line 1144)
        main
        Ply file does not contain normals
[WARNING] PoissonRecon failed with 7 threads, let's retry with 3...
[INFO]    running "/code/SuperBuild/install/bin/PoissonRecon" --in "/code/odm_filterpoints/point_cloud.ply" --out "/code/odm_meshing/odm_mesh.dirty.ply" --depth 10 --pointWeight 4.0 --samplesPerNode 1.0 --threads 7 --bType 2 --linearFit
[ERROR] Src/PoissonRecon.cpp (Line 1144)
        main
        Ply file does not contain normals
[WARNING] PoissonRecon failed with 3 threads, let's retry with 1...
[INFO]    running "/code/SuperBuild/install/bin/PoissonRecon" --in "/code/odm_filterpoints/point_cloud.ply" --out "/code/odm_meshing/odm_mesh.dirty.ply" --depth 10 --pointWeight 4.0 --samplesPerNode 1.0 --threads 3 --bType 2 --linearFit
[ERROR] Src/PoissonRecon.cpp (Line 1144)
        main
        Ply file does not contain normals
[WARNING] PoissonRecon failed with 1 threads, let's retry with 0...
[INFO]    running "/code/SuperBuild/install/bin/PoissonRecon" --in "/code/odm_filterpoints/point_cloud.ply" --out "/code/odm_meshing/odm_mesh.dirty.ply" --depth 10 --pointWeight 4.0 --samplesPerNode 1.0 --threads 1 --bType 2 --linearFit
[ERROR] Src/PoissonRecon.cpp (Line 1144)
        main
        Ply file does not contain normals
[INFO]    running "/code/SuperBuild/install/bin/OpenMVS/ReconstructMesh" -i "/code/odm_meshing/odm_mesh.dirty.ply" -o "/code/odm_meshing/odm_mesh.ply" --remove-spikes 0 --remove-spurious 20 --smooth 0 --target-face-num 600000 -v 0
/code/SuperBuild/install/bin/OpenMVS/ReconstructMesh: error while loading shared libraries: libcuda.so.1: cannot open shared object file: No such file or directory

===== Dumping Info for Geeks (developers need this to fix bugs) =====
Child returned 127
Traceback (most recent call last):
  File "/code/stages/odm_app.py", line 94, in execute
    self.first_stage.run()
  File "/code/opendm/types.py", line 346, in run
    self.next_stage.run(outputs)
  File "/code/opendm/types.py", line 346, in run
    self.next_stage.run(outputs)
  File "/code/opendm/types.py", line 346, in run
    self.next_stage.run(outputs)
  [Previous line repeated 2 more times]
  File "/code/opendm/types.py", line 327, in run
    self.process(self.args, outputs)
  File "/code/stages/odm_meshing.py", line 24, in process
    mesh.screened_poisson_reconstruction(tree.filtered_point_cloud,
  File "/code/opendm/mesh.py", line 208, in screened_poisson_reconstruction
    system.run('"{reconstructmesh}" -i "{infile}" '
  File "/code/opendm/system.py", line 106, in run
    raise SubprocessException("Child returned {}".format(retcode), retcode)
opendm.system.SubprocessException: Child returned 127

===== Done, human-readable information to follow... =====

[ERROR]   The program exited with a strange error code. Please report it at https://community.opendronemap.org

I am running it with the following command:
C:\Users\u123>docker run -it --rm -v "d:\site2:/code/images" -v "d:\site2\orthophoto:/code/odm_orthophoto" -v "d:\site2\georeferencing:/code/odm_georeferencing" -v "d:\site2\report:/code/odm_report" opendronemap/odm:gpu --fast-orthophoto --pc-quality high --pc-filter 5 --pc-geometric --pc-rectify --resize-to -1 --use-3dmesh --matcher-neighbors 12 --mesh-octree-depth 7 --mesh-size 300000 --orthophoto-resolution 1 --auto-boundary --feature-quality high --dem-resolution 2 --dsm

Saijin_Naib · 26 March 2022 02:16

Would you mind re-running that dataset with the same parameters on the non-GPU Docker image and posting the log/report so we can learn a bit more about the dataset and how it reconstructs (or not)?

ChrisDAT · 26 March 2022 08:59

Yes, i will do. But i thought its just a library missing:

ChrisDAT · 26 March 2022 13:05

I did two runs, one with GPU and one without. the output of the runs are both in this zip file:

And the dataset is located here:
https://drive.google.com/drive/folders/1rn5cPC3Dx98Dm7ZO7zA8AiLaBIIcz7Fn?usp=sharing

Program arguments have been on both calls:
docker run -it --rm -v "F:\FH\code\site2-wide\images:/code/images" -v "F:\FH\code\site2-wide\odm_orthophoto:/code/odm_orthophoto" -v "F:\FH\code\site2-wide\odm_georeferencing:/code/odm_georeferencing" -v "F:\FH\code\site2-wide\report:/code/odm_report" opendronemap/odm --fast-orthophoto --pc-quality high --pc-filter 5 --pc-geometric --pc-rectify --resize-to -1 --use-3dmesh --matcher-neighbors 12 --mesh-octree-depth 10 --mesh-size 300000 --orthophoto-resolution 1 --auto-boundary --feature-quality medium --dem-resolution 2 --dsm > odm.out 2>&1

And thank you very much for your time

system · 25 April 2022 13:06

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.

Saijin_Naib · 24 May 2022 21:34

Are you able to try on the latest/current releases? We’ve made a number of significant fixes that might help.

I can’t reproduce locally (no CUDA-capable hardware), but I got a good, if bowled, reconstruction:

Might be fixable with a more constrained --gps-accuracy if you have a good mean/estimated GPS accuracy for the survey.

Edit: --gps-accuracy 1.0

Saijin_Naib · 24 May 2022 21:38