Hi community,
This is a continuation of a previous discussion…
I am attempting to process 65k images over an area with buildings and a few trees.
Images were captured using a drone flying in 8 directions with 80% overlap and at a height of 35m above the ground.
I gained access to a second VM. So now I have two VMs with 256GB RAM and 256GB SWAP in each
I have set up NodeODM on the 2 machines and ClusterODM on one of them.
My first test with a sample of 250 images is running but seems to be only using one node. I used the defaults…
Info from tasks.json:
Maybe it is only using one node as it does not require more than that?
Or I need to tell it to use both nodes…
Any advice is appreciated!
I hope to start it with approx 8k images once the first test has completed.
Does ClusterODM only work with a split merge? In that case, I would not get the textured mesh. Is this correct?
So…
I just ran cluster ODM on 4k images with a split of 2000 and a split-overlap of 25
Going to try again with 2000 images and a split of 800.
THIS failed with the following error… (Any advice is appreciated)
[INFO] running "/code/SuperBuild/install/bin/opensfm/bin/opensfm" export_visualsfm --points "/var/www/data/8ebe40ef-be26-4fbf-a2cd-5acf352f9f16/opensfm"
[INFO] Finished opensfm stage
[INFO] Running openmvs stage
[INFO] running "/code/SuperBuild/install/bin/opensfm/bin/opensfm" export_openmvs "/var/www/data/8ebe40ef-be26-4fbf-a2cd-5acf352f9f16/opensfm"
[INFO] Depthmap resolution set to: 507px
[INFO] Running dense reconstruction. This might take a while.
[INFO] Estimating depthmaps
[INFO] nvidia-smi not found in PATH, using CPU
[INFO] running "/code/SuperBuild/install/bin/OpenMVS/DensifyPointCloud" "/var/www/data/8ebe40ef-be26-4fbf-a2cd-5acf352f9f16/opensfm/undistorted/openmvs/scene.mvs" --resolution-level 2 --dense-config-file "/var/www/data/8ebe40ef-be26-4fbf-a2cd-5acf352f9f16/opensfm/undistorted/openmvs/Densify.ini" --min-resolution 507 --max-resolution 2635 --max-threads 32 --number-views-fuse 2 --sub-resolution-levels 2 -w "/var/www/data/8ebe40ef-be26-4fbf-a2cd-5acf352f9f16/opensfm/undistorted/openmvs/depthmaps" -v 0 --cuda-device -2 --geometric-iters 0
20:44:39 [App ] Build date: Jun 24 2022, 23:48:50
20:44:39 [App ] CPU: Intel(R) Xeon(R) Gold 6150 CPU @ 2.70GHz (32 cores)
20:44:39 [App ] RAM: 251.73GB Physical Memory 256.00GB Virtual Memory
20:44:39 [App ] OS: Linux 3.10.0-1160.66.1.el7.x86_64 (x86_64)
20:44:39 [App ] Disk: 485.98GB (511.75GB) space
20:44:39 [App ] SSE & AVX compatible CPU & OS detected
20:44:39 [App ] Command line: DensifyPointCloud /var/www/data/8ebe40ef-be26-4fbf-a2cd-5acf352f9f16/opensfm/undistorted/openmvs/scene.mvs --resolution-level 2 --dense-config-file /var/www/data/8ebe40ef-be26-4fbf-a2cd-5acf352f9f16/opensfm/undistorted/openmvs/Densify.ini --min-resolution 507 --max-resolution 2635 --max-threads 32 --number-views-fuse 2 --sub-resolution-levels 2 -w /var/www/data/8ebe40ef-be26-4fbf-a2cd-5acf352f9f16/opensfm/undistorted/openmvs/depthmaps -v 0 --cuda-device -2 --geometric-iters 0
20:44:42 [App ] Find a camera not pointing towards the scene center; the scene will be considered unbounded (no ROI)
20:44:53 [App ] Preparing images for dense reconstruction completed: 2666 images (10s894ms)
20:45:37 [App ] Selecting images for dense reconstruction completed: 0 images (44s277ms)
Fused depth-maps 0 (100%, 0ms)
20:45:38 [App ] Densifying point-cloud completed: 0 points (55s666ms)
20:45:38 [App ] MEMORYINFO: {
20:45:38 [App ] VmPeak: 7329912 kB
20:45:38 [App ] VmSize: 6290764 kB
20:45:38 [App ] } ENDINFO
[INFO] running "/code/SuperBuild/install/bin/OpenMVS/DensifyPointCloud" --filter-point-cloud -1 -i "/var/www/data/8ebe40ef-be26-4fbf-a2cd-5acf352f9f16/opensfm/undistorted/openmvs/scene_dense.mvs" -v 0 --cuda-device -2 --geometric-iters 0
20:45:38 [App ] Build date: Jun 24 2022, 23:48:50
20:45:38 [App ] CPU: Intel(R) Xeon(R) Gold 6150 CPU @ 2.70GHz (32 cores)
20:45:38 [App ] RAM: 251.73GB Physical Memory 256.00GB Virtual Memory
20:45:38 [App ] OS: Linux 3.10.0-1160.66.1.el7.x86_64 (x86_64)
20:45:38 [App ] Disk: 50.46GB (69.98GB) space
20:45:38 [App ] SSE & AVX compatible CPU & OS detected
20:45:38 [App ] Command line: DensifyPointCloud --filter-point-cloud -1 -i /var/www/data/8ebe40ef-be26-4fbf-a2cd-5acf352f9f16/opensfm/undistorted/openmvs/scene_dense.mvs -v 0 --cuda-device -2 --geometric-iters 0
20:45:38 [App ] error: no valid point-cloud for the ROI estimation
===== Dumping Info for Geeks (developers need this to fix bugs) =====
Child returned 1
Traceback (most recent call last):
File "/code/stages/odm_app.py", line 94, in execute
self.first_stage.run()
File "/code/opendm/types.py", line 347, in run
self.next_stage.run(outputs)
File "/code/opendm/types.py", line 347, in run
self.next_stage.run(outputs)
File "/code/opendm/types.py", line 347, in run
self.next_stage.run(outputs)
[Previous line repeated 1 more time]
File "/code/opendm/types.py", line 328, in run
self.process(self.args, outputs)
File "/code/stages/openmvs.py", line 206, in process
system.run('"%s" %s' % (context.omvs_densify_path, ' '.join(config + gpu_config + extra_config)))
File "/code/opendm/system.py", line 106, in run
raise SubprocessException("Child returned {}".format(retcode), retcode)
opendm.system.SubprocessException: Child returned 1
===== Done, human-readable information to follow... =====
[ERROR] Uh oh! Processing stopped because of strange values in the reconstruction. This is often a sign that the input data has some issues or the software cannot deal with it. Have you followed best practices for data acquisition? See https://docs.opendronemap.org/flying/