Error: Mesh is empty!

gast · 7 April 2020 13:14

Hi everyone,

I am starting a new topic on the above after searching the community for while and not finding an answer that fixed the errors…
The above error seemed to have occurred and reported in several cases before, yet a solution still seems elusive as far as I can tell. It seems to buried somewhere in the mve step, but I have not found where yet.

Examples of earlier reports in the community

Issues trying to process IR images Mavic 2ED - Help please not the first time all images are geotagged
Additional processing time with GCPs
Error during processing 128 images on 15GB and 7 threads
Error "Cannot determine image format" - #4 by Jesus_Flores

I am trying to solve a sample of multi-spectral (multi-camera) set attached below:

Link to data

https://drive.google.com/open?id=1z7Z40eUsIV_DF0pEQi47kPGL9FA4wz2i

I keep crashing at the MVE stage, mostly with , Error: Mesh is empty! which is preceded by Reading PLY: 0 verts... done. message (0 vertices PLY file…)
Although I am using multi-spectral (camera) set, seems ODM is identifying it properly since these lines are written at the beginning of the log/run.
[INFO] Increasing matcher neighbors to 24 to accomodate multi-camera setup [INFO] Multi-camera setup, using BOW matching

These errors appear whether I use ODM or WebODM ( I did use ./webodm update). I am running all these on Ubuntu 18.04LTS with 32GB RAM and 500GB disk.

I have tried several parameters but usually I get the error below.

Last lines of logs in Mesh is empty error

[INFO] Running mve stage
[INFO] running /code/SuperBuild/src/elibs/mve/apps/makescene/makescene “/datasets/code/opensfm/undistorted/reconstruction_blue.nvm” “/datasets/code/mve”
MVE Makescene (built on Mar 19 2020, 15:23:11)
Info: Detected VisualSFM bundle format.
NVM: Loading file…
NVM: Number of views: 6
NVM: Number of features: 317
Creating output directories…
Writing MVE views…
Writing MVE view: view_0004.mve…
Writing MVE view: view_0005.mve…
Writing MVE view: view_0002.mve…
Writing MVE view: view_0003.mve…
Writing MVE view: view_0001.mve…
Writing MVE view: view_0000.mve…
Writing bundle file…
Writing bundle (6 cameras, 317 features): /datasets/code/mve/synth_0.out…

Done importing NVM file!
[INFO] Running dense reconstruction. This might take a while.
[INFO] running /code/SuperBuild/src/elibs/mve/apps/dmrecon/dmrecon -s3 --progress=fancy --local-neighbors=2 “/datasets/code/mve”
MVE Depth Map Reconstruction (built on Mar 19 2020, 15:23:07)
Initializing scene with 6 views…
Initialized 6 views (max ID is 5), took 0ms.
Reading Photosynther file (6 cameras, 317 features)…
Reconstructing all views…
0 of 6 completed (0.00%)
Global View Selection failed
6 of 6 completed (100.00%)
Reconstruction took 2903ms.
Saving views back to disc…
Saving views to MVE files… done.
[INFO] running /code/SuperBuild/src/elibs/mve/apps/scene2pset/scene2pset -F3 “/datasets/code/mve” “/datasets/code/mve/mve_dense_point_cloud.ply”
MVE Scene to Pointset (built on Mar 19 2020, 15:23:20)
Using depthmap “depth-L3” and color image “undist-L3”
Initializing scene with 6 views…
Initialized 6 views (max ID is 5), took 0ms.
Processing view “0002” (with colors)…
Processing view “0005” (with colors)…
Processing view “0001” (with colors)…
Processing view “0000” (with colors)…
Processing view “0003” (with colors)…
Writing final point set (0 points)…
Writing PLY file (0 verts, with colors, with normals, with confidences, with values, 0 faces)… done.
[INFO] running /code/SuperBuild/src/elibs/mve/apps/meshclean/meshclean -t0.6 --no-clean --component-size=0 “/datasets/code/mve/mve_dense_point_cloud.ply” “/datasets/code/mve/mve_dense_point_cloud.filtered.ply”
MVE FSSR Mesh Cleaning (built on Mar 19 2020, 15:23:46)
Loading mesh: /datasets/code/mve/mve_dense_point_cloud.ply
PLY Loader: comment Export generated by libmve
Reading PLY: 0 verts… done.
Error: Mesh is empty!
Traceback (most recent call last):
File “/code/run.py”, line 61, in
app.execute()
File “/code/stages/odm_app.py”, line 92, in execute
self.first_stage.run()
File “/code/opendm/types.py”, line 344, in run
self.next_stage.run(outputs)
File “/code/opendm/types.py”, line 344, in run
self.next_stage.run(outputs)
File “/code/opendm/types.py”, line 344, in run
self.next_stage.run(outputs)
File “/code/opendm/types.py”, line 344, in run
self.next_stage.run(outputs)
File “/code/opendm/types.py”, line 325, in run
self.process(self.args, outputs)
File “/code/stages/mve.py”, line 89, in process
system.run(‘%s -t%s --no-clean --component-size=0 “%s” “%s”’ % (context.meshclean_path, min(1.0, args.mve_confidence), tree.mve_model, mve_filtered_model), env_vars={‘OMP_NUM_THREADS’: args.max_concurrency})
File “/code/opendm/system.py”, line 76, in run
raise Exception(“Child returned {}”.format(retcode))
Exception: Child returned 1

Other options I have tried are listed below. Not all errors are identical but all seem to appear in the MVE stage.

opendronemap/odm:latest , mentioned here

Last lines of log

[INFO] Running mve stage
[INFO] running /code/SuperBuild/src/elibs/mve/apps/makescene/makescene “/datasets/code/opensfm/undistorted/reconstruction_blue.nvm” “/datasets/code/mve”
MVE Makescene (built on Mar 19 2020, 15:23:11)
Info: Detected VisualSFM bundle format.
NVM: Loading file…
NVM: Number of views: 6
NVM: Number of features: 143
Creating output directories…
Writing MVE views…
Writing MVE view: view_0005.mve…
Writing MVE view: view_0004.mve…
Writing MVE view: view_0003.mve…
Writing MVE view: view_0002.mve…
Writing MVE view: view_0001.mve…
Writing MVE view: view_0000.mve…
Writing bundle file…
Writing bundle (6 cameras, 143 features): /datasets/code/mve/synth_0.out…

Done importing NVM file!
[INFO] Running dense reconstruction. This might take a while.
[INFO] running /code/SuperBuild/src/elibs/mve/apps/dmrecon/dmrecon -s3 --progress=fancy --local-neighbors=2 “/datasets/code/mve”
MVE Depth Map Reconstruction (built on Mar 19 2020, 15:23:07)
Initializing scene with 6 views…
Initialized 6 views (max ID is 5), took 0ms.
Reading Photosynther file (6 cameras, 143 features)…
Reconstructing all views…
Global View Selection failed
1 of 6 completed (16.67%)
6 of 6 completed (100.00%)
Reconstruction took 2762ms.
Saving views back to disc…
Saving views to MVE files… done.
[INFO] running /code/SuperBuild/src/elibs/mve/apps/scene2pset/scene2pset -F3 “/datasets/code/mve” “/datasets/code/mve/mve_dense_point_cloud.ply”
MVE Scene to Pointset (built on Mar 19 2020, 15:23:20)
Using depthmap “depth-L3” and color image “undist-L3”
Initializing scene with 6 views…
Initialized 6 views (max ID is 5), took 0ms.
Processing view “0002” (with colors)…
Processing view “0004” (with colors)…
Processing view “0003” (with colors)…
Processing view “0005” (with colors)…
Processing view “0001” (with colors)…
Writing final point set (0 points)…
Writing PLY file (0 verts, with colors, with normals, with confidences, with values, 0 faces)… done.
[INFO] running /code/SuperBuild/src/elibs/mve/apps/meshclean/meshclean -t0.6 --no-clean --component-size=0 “/datasets/code/mve/mve_dense_point_cloud.ply” “/datasets/code/mve/mve_dense_point_cloud.filtered.ply”
MVE FSSR Mesh Cleaning (built on Mar 19 2020, 15:23:46)
Loading mesh: /datasets/code/mve/mve_dense_point_cloud.ply
PLY Loader: comment Export generated by libmve
Reading PLY: 0 verts… done.
Error: Mesh is empty!
Traceback (most recent call last):
File “/code/run.py”, line 61, in
app.execute()
File “/code/stages/odm_app.py”, line 92, in execute
self.first_stage.run()
File “/code/opendm/types.py”, line 344, in run
self.next_stage.run(outputs)
File “/code/opendm/types.py”, line 344, in run
self.next_stage.run(outputs)
File “/code/opendm/types.py”, line 344, in run
self.next_stage.run(outputs)
File “/code/opendm/types.py”, line 344, in run
self.next_stage.run(outputs)
File “/code/opendm/types.py”, line 325, in run
self.process(self.args, outputs)
File “/code/stages/mve.py”, line 89, in process
system.run(‘%s -t%s --no-clean --component-size=0 “%s” “%s”’ % (context.meshclean_path, min(1.0, args.mve_confidence), tree.mve_model, mve_filtered_model), env_vars={‘OMP_NUM_THREADS’: args.max_concurrency})
File “/code/opendm/system.py”, line 76, in run
raise Exception(“Child returned {}”.format(retcode))
Exception: Child returned 1

--use-opensfm-dense, mentioned here as well

Last lines of log

[INFO] Finished opensfm stage
[INFO] Running odm_filterpoints stage
[INFO] Filtering point cloud (statistical, meanK 16, standard deviation 2.5)
[INFO] running /code/build/bin/odm_filterpoints -inputFile /datasets/code/opensfm/undistorted/depthmaps/merged.ply -outputFile /datasets/code/odm_filterpoints/point_cloud.ply -sd 2.5 -meank 16 -sample 0
[INFO] Finished odm_filterpoints stage
[INFO] Running odm_meshing stage
[INFO] Writing ODM Mesh file in: /datasets/code/odm_meshing/odm_mesh.ply
[INFO] running /code/SuperBuild/src/PoissonRecon/Bin/Linux/PoissonRecon --in /datasets/code/odm_filterpoints/point_cloud.ply --out /datasets/code/odm_meshing/odm_mesh.dirty.ply --depth 10 --pointWeight 4 --samplesPerNode 1.0 --threads 7 --linearFit
[INFO] running /code/build/bin/odm_cleanmesh -inputFile /datasets/code/odm_meshing/odm_mesh.dirty.ply -outputFile /datasets/code/odm_meshing/odm_mesh.ply -removeIslands -decimateMesh 200000
ERROR: In /build/vtk6-YpT4yb/vtk6-6.2.0+dfsg1/IO/PLY/vtkPLYWriter.cxx, line 114
vtkPLYWriter (0xc16e60): No data to write!

[INFO] Writing ODM 2.5D Mesh file in: /datasets/code/odm_meshing/odm_25dmesh.ply
[WARNING] Cannot calculate GSD, using requested resolution of 5.0
[INFO] ODM 2.5D DSM resolution: 0.2
[INFO] Created temporary directory: /datasets/code/odm_meshing/tmp
[INFO] Creating DSM for 2.5D mesh
[INFO] running pdal info /datasets/code/odm_filterpoints/point_cloud.ply > /tmp/tmpl4B24Y.json
Traceback (most recent call last):
File “/code/run.py”, line 61, in
app.execute()
File “/code/stages/odm_app.py”, line 92, in execute
self.first_stage.run()
File “/code/opendm/types.py”, line 344, in run
self.next_stage.run(outputs)
File “/code/opendm/types.py”, line 344, in run
self.next_stage.run(outputs)
File “/code/opendm/types.py”, line 344, in run
self.next_stage.run(outputs)
File “/code/opendm/types.py”, line 344, in run
self.next_stage.run(outputs)
File “/code/opendm/types.py”, line 344, in run
self.next_stage.run(outputs)
File “/code/opendm/types.py”, line 325, in run
self.process(self.args, outputs)
File “/code/stages/odm_meshing.py”, line 75, in process
smooth_dsm=not args.fast_orthophoto)
File “/code/opendm/mesh.py”, line 35, in create_25dmesh
apply_smoothing=smooth_dsm
File “/code/opendm/dem/commands.py”, line 92, in create_dem
extent = point_cloud.get_extent(input_point_cloud)
File “/code/opendm/point_cloud.py”, line 90, in get_extent
if bbox is None: raise Exception(“Cannot compute bounds for %s (bbox key missing)” % input_point_cloud)
Exception: Cannot compute bounds for /datasets/code/odm_filterpoints/point_cloud.ply (bbox key missing)

--mve-confidence 0.3 - (default is 0.6)

Last lines of log

/usr/local/lib/python2.7/dist-packages/numpy/core/fromnumeric.py:2920: RuntimeWarning: Mean of empty slice.
out=out, **kwargs)
/usr/local/lib/python2.7/dist-packages/numpy/core/_methods.py:85: RuntimeWarning: invalid value encountered in double_scalars
ret = ret.dtype.type(ret / rcount)
Traceback (most recent call last):
File “/code/SuperBuild/src/opensfm/bin/opensfm”, line 34, in
command.run(args)
File “/code/SuperBuild/src/opensfm/opensfm/commands/reconstruct.py”, line 23, in run
incremental_reconstruction(data, graph)
File “/code/SuperBuild/src/opensfm/opensfm/reconstruction.py”, line 1443, in incremental_reconstruction
data, graph, graph_inliers, reconstruction, remaining_images, camera_priors, gcp)
File “/code/SuperBuild/src/opensfm/opensfm/reconstruction.py”, line 1349, in grow_reconstruction
remove_outliers(graph_inliers, reconstruction, config)
File “/code/SuperBuild/src/opensfm/opensfm/reconstruction.py”, line 1144, in remove_outliers
threshold = get_actual_threshold(config, reconstruction.points)
File “/code/SuperBuild/src/opensfm/opensfm/reconstruction.py”, line 1131, in get_actual_threshold
mean, std = get_error_distribution(points)
File “/code/SuperBuild/src/opensfm/opensfm/reconstruction.py”, line 1122, in get_error_distribution
robust_std = 1.486*np.median(np.linalg.norm(all_errors-robust_mean, axis=1))
File “/usr/local/lib/python2.7/dist-packages/numpy/linalg/linalg.py”, line 2390, in norm
return sqrt(add.reduce(s, axis=axis, keepdims=keepdims))
numpy.core._internal.AxisError: axis 1 is out of bounds for array of dimension 1
Traceback (most recent call last):
File “/code/run.py”, line 61, in
app.execute()
File “/code/stages/odm_app.py”, line 92, in execute
self.first_stage.run()
File “/code/opendm/types.py”, line 344, in run
self.next_stage.run(outputs)
File “/code/opendm/types.py”, line 344, in run
self.next_stage.run(outputs)
File “/code/opendm/types.py”, line 344, in run
self.next_stage.run(outputs)
File “/code/opendm/types.py”, line 325, in run
self.process(self.args, outputs)
File “/code/stages/run_opensfm.py”, line 30, in process
octx.reconstruct(self.rerun())
File “/code/opendm/osfm.py”, line 41, in reconstruct
self.run(‘reconstruct’)
File “/code/opendm/osfm.py”, line 23, in run
(context.opensfm_path, command, self.opensfm_project_path))
File “/code/opendm/system.py”, line 76, in run
raise Exception(“Child returned {}”.format(retcode))
Exception: Child returned 1

--opensfm-depthmap-method BRUTE_FORCE, (instead of PATCH_MATCH)

Last lines of log

[INFO] Finished opensfm stage
[INFO] Running mve stage
[INFO] running /code/SuperBuild/src/elibs/mve/apps/makescene/makescene “/datasets/code/opensfm/undistorted/reconstruction_blue.nvm” “/datasets/code/mve”
MVE Makescene (built on Mar 19 2020, 15:23:11)
Info: Detected VisualSFM bundle format.
NVM: Loading file…
NVM: Number of views: 9
NVM: Number of features: 1325
Creating output directories…
Writing MVE views…
Writing MVE view: view_0001.mve…
Writing MVE view: view_0007.mve…
Writing MVE view: view_0005.mve…
Writing MVE view: view_0004.mve…
Writing MVE view: view_0006.mve…
Writing MVE view: view_0000.mve…
Writing MVE view: view_0002.mve…
Writing MVE view: view_0003.mve…
Writing MVE view: view_0008.mve…
Writing bundle file…
Writing bundle (9 cameras, 1325 features): /datasets/code/mve/synth_0.out…

Done importing NVM file!
[INFO] Running dense reconstruction. This might take a while.
[INFO] running /code/SuperBuild/src/elibs/mve/apps/dmrecon/dmrecon -s3 --progress=fancy --local-neighbors=2 “/datasets/code/mve”
MVE Depth Map Reconstruction (built on Mar 19 2020, 15:23:07)
Initializing scene with 9 views…
Initialized 9 views (max ID is 8), took 0ms.
Reading Photosynther file (9 cameras, 1325 features)…
Reconstructing all views…
0 of 9 completed (0.00%)
*** Error in `/code/SuperBuild/src/elibs/mve/apps/dmrecon/dmrecon’: corrupted double-linked list: 0x0000000000f82070 ***
======= Backtrace: =========
/lib/x86_64-linux-gnu/libc.so.6(+0x777e5)[0x7fc59a0f17e5]
/lib/x86_64-linux-gnu/libc.so.6(+0x82970)[0x7fc59a0fc970]
/lib/x86_64-linux-gnu/libc.so.6(__libc_malloc+0x54)[0x7fc59a0fe184]
/usr/lib/x86_64-linux-gnu/libstdc++.so.6(_Znwm+0x18)[0x7fc59ae2fe78]
/usr/lib/x86_64-linux-gnu/libstdc++.so.6(_ZNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEE9_M_mutateEmmPKcm+0x69)[0x7fc59aec1499]
/usr/lib/x86_64-linux-gnu/libstdc++.so.6(_ZNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEE9_M_appendEPKcm+0x63)[0x7fc59aec2833]
/code/SuperBuild/src/elibs/mve/apps/dmrecon/dmrecon[0x433082]
/code/SuperBuild/src/elibs/mve/apps/dmrecon/dmrecon[0x433548]
/code/SuperBuild/src/elibs/mve/apps/dmrecon/dmrecon[0x40bec3]
/usr/lib/x86_64-linux-gnu/libgomp.so.1(GOMP_parallel+0x3f)[0x7fc59a882cbf]
/code/SuperBuild/src/elibs/mve/apps/dmrecon/dmrecon[0x4083d2]
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf0)[0x7fc59a09a830]
/code/SuperBuild/src/elibs/mve/apps/dmrecon/dmrecon[0x409669]
======= Memory map: ========

"I excluded the memory map part of the log here

7fc59882b000-7fc59902b000 rw-p 00000000 00:00 0
7fc59902b000-7fc59902c000 —p 00000000 00:00 0
7fc59902c000-7fc59982c000 rw-p 00000000 00:00 0
7fc59982c000-7fc59982f000 r-xp 00000000 103:03 23727256 /lib/x86_64-linux-gnu/libdl-2.23.so
7fc59982f000-7fc599a2e000 —p 00003000 103:03 23727256 /lib/x86_64-linux-gnu/libdl-2.23.so
7fc599a2e000-7fc599a2f000 r–p 00002000 103:03 23727256 /lib/x86_64-linux-gnu/libdl-2.23.so
7fc599a2f000-7fc599a30000 rw-p 00003000 103:03 23727256 /lib/x86_64-linux-gnu/libdl-2.23.so
7fc599a30000-7fc599a3b000 r-xp 00000000 103:03 24249642 /usr/lib/x86_64-linux-gnu/libjbig.so.0
7fc599a3b000-7fc599c3a000 —p 0000b000 103:03 24249642 /usr/lib/x86_64-linux-gnu/libjbig.so.0
7fc599c3a000-7fc599c3b000 r–p 0000a000 103:03 24249642 /usr/lib/x86_64-linux-gnu/libjbig.so.0
7fc599c3b000-7fc599c3e000 rw-p 0000b000 103:03 24249642 /usr/lib/x86_64-linux-gnu/libjbig.so.0
7fc599c3e000-7fc599c5f000 r-xp 00000000 103:03 23202649 /lib/x86_64-linux-gnu/liblzma.so.5.0.0
7fc599c5f000-7fc599e5e000 —p 00021000 103:03 23202649 /lib/x86_64-linux-gnu/liblzma.so.5.0.0
7fc599e5e000-7fc599e5f000 r–p 00020000 103:03 23202649 /lib/x86_64-linux-gnu/liblzma.so.5.0.0
7fc599e5f000-7fc599e60000 rw-p 00021000 103:03 23202649 /lib/x86_64-linux-gnu/liblzma.so.5.0.0
7fc599e60000-7fc599e79000 r-xp 00000000 103:03 23727302 /lib/x86_64-linux-gnu/libz.so.1.2.8
7fc599e79000-7fc59a078000 —p 00019000 103:03 23727302 /lib/x86_64-linux-gnu/libz.so.1.2.8
7fc59a078000-7fc59a079000 r–p 00018000 103:03 23727302 /lib/x86_64-linux-gnu/libz.so.1.2.8
7fc59a079000-7fc59a07a000 rw-p 00019000 103:03 23727302 /lib/x86_64-linux-gnu/libz.so.1.2.8
7fc59a07a000-7fc59a23a000 r-xp 00000000 103:03 23727247 /lib/x86_64-linux-gnu/libc-2.23.so
7fc59a23a000-7fc59a43a000 —p 001c0000 103:03 23727247 /lib/x86_64-linux-gnu/libc-2.23.so
7fc59a43a000-7fc59a43e000 r–p 001c0000 103:03 23727247 /lib/x86_64-linux-gnu/libc-2.23.so
7fc59a43e000-7fc59a440000 rw-p 001c4000 103:03 23727247 /lib/x86_64-linux-gnu/libc-2.23.so
7fc59a440000-7fc59a444000 rw-p 00000000 00:00 0
7fc59a444000-7fc59a45c000 r-xp 00000000 103:03 23727288 /lib/x86_64-linux-gnu/libpthread-2.23.so
7fc59a45c000-7fc59a65b000 —p 00018000 103:03 23727288 /lib/x86_64-linux-gnu/libpthread-2.23.so
7fc59a65b000-7fc59a65c000 r–p 00017000 103:03 23727288 /lib/x86_64-linux-gnu/libpthread-2.23.so
7fc59a65c000-7fc59a65d000 rw-p 00018000 103:03 23727288 /lib/x86_64-linux-gnu/libpthread-2.23.so
7fc59a65d000-7fc59a661000 rw-p 00000000 00:00 0
7fc59a661000-7fc59a677000 r-xp 00000000 103:03 23202639 /lib/x86_64-linux-gnu/libgcc_s.so.1
7fc59a677000-7fc59a876000 —p 00016000 103:03 23202639 /lib/x86_64-linux-gnu/libgcc_s.so.1
7fc59a876000-7fc59a877000 rw-p 00015000 103:03 23202639 /lib/x86_64-linux-gnu/libgcc_s.so.1
7fc59a877000-7fc59a898000 r-xp 00000000 103:03 24249496 /usr/lib/x86_64-linux-gnu/libgomp.so.1.0.0
7fc59a898000-7fc59aa97000 —p 00021000 103:03 24249496 /usr/lib/x86_64-linux-gnu/libgomp.so.1.0.0
7fc59aa97000-7fc59aa98000 r–p 00020000 103:03 24249496 /usr/lib/x86_64-linux-gnu/libgomp.so.1.0.0
7fc59aa98000-7fc59aa99000 rw-p 00021000 103:03 24249496 /usr/lib/x86_64-linux-gnu/libgomp.so.1.0.0
7fc59aa99000-7fc59aba1000 r-xp 00000000 103:03 23727266 /lib/x86_64-linux-gnu/libm-2.23.so
7fc59aba1000-7fc59ada0000 —p 00108000 103:03 23727266 /lib/x86_64-linux-gnu/libm-2.23.so
7fc59ada0000-7fc59ada1000 r–p 00107000 103:03 23727266 /lib/x86_64-linux-gnu/libm-2.23.so
7fc59ada1000-7fc59ada2000 rw-p 00108000 103:03 23727266 /lib/x86_64-linux-gnu/libm-2.23.so
7fc59ada2000-7fc59af14000 r-xp 00000000 103:03 24249901 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.21
7fc59af14000-7fc59b114000 —p 00172000 103:03 24249901 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.21
7fc59b114000-7fc59b11e000 r–p 00172000 103:03 24249901 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.21
7fc59b11e000-7fc59b120000 rw-p 0017c000 103:03 24249901 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.21
7fc59b120000-7fc59b124000 rw-p 00000000 00:00 0
7fc59b124000-7fc59b17b000 r-xp 00000000 103:03 24249646 /usr/lib/x86_64-linux-gnu/libjpeg.so.8.0.2
7fc59b17b000-7fc59b37b000 —p 00057000 103:03 24249646 /usr/lib/x86_64-linux-gnu/libjpeg.so.8.0.2
7fc59b37b000-7fc59b37c000 r–p 00057000 103:03 24249646 /usr/lib/x86_64-linux-gnu/libjpeg.so.8.0.2
7fc59b37c000-7fc59b37d000 rw-p 00058000 103:03 24249646 /usr/lib/x86_64-linux-gnu/libjpeg.so.8.0.2
7fc59b37d000-7fc59b3ee000 r-xp 00000000 103:03 24249957 /usr/lib/x86_64-linux-gnu/libtiff.so.5.2.4
7fc59b3ee000-7fc59b5ee000 —p 00071000 103:03 24249957 /usr/lib/x86_64-linux-gnu/libtiff.so.5.2.4
7fc59b5ee000-7fc59b5ef000 r–p 00071000 103:03 24249957 /usr/lib/x86_64-linux-gnu/libtiff.so.5.2.4
7fc59b5ef000-7fc59b5f2000 rw-p 00072000 103:03 24249957 /usr/lib/x86_64-linux-gnu/libtiff.so.5.2.4
7fc59b5f2000-7fc59b616000 r-xp 00000000 103:03 23727287 /lib/x86_64-linux-gnu/libpng12.so.0.54.0
7fc59b616000-7fc59b815000 —p 00024000 103:03 23727287 /lib/x86_64-linux-gnu/libpng12.so.0.54.0
7fc59b815000-7fc59b816000 r–p 00023000 103:03 23727287 /lib/x86_64-linux-gnu/libpng12.so.0.54.0
7fc59b816000-7fc59b817000 rw-p 00024000 103:03 23727287 /lib/x86_64-linux-gnu/libpng12.so.0.54.0
7fc59b817000-7fc59b83d000 r-xp 00000000 103:03 23727237 /lib/x86_64-linux-gnu/ld-2.23.so
7fc59ba0b000-7fc59ba15000 rw-p 00000000 00:00 0
7fc59ba3b000-7fc59ba3c000 rw-p 00000000 00:00 0
7fc59ba3c000-7fc59ba3d000 r–p 00025000 103:03 23727237 /lib/x86_64-linux-gnu/ld-2.23.so
7fc59ba3d000-7fc59ba3e000 rw-p 00026000 103:03 23727237 /lib/x86_64-linux-gnu/ld-2.23.so
7fc59ba3e000-7fc59ba3f000 rw-p 00000000 00:00 0
7ffc39f10000-7ffc39f31000 rw-p 00000000 00:00 0 [stack]
7ffc39f86000-7ffc39f89000 r–p 00000000 00:00 0 [vvar]
7ffc39f89000-7ffc39f8b000 r-xp 00000000 00:00 0 [vdso]
ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall]
Aborted (core dumped)
[WARNING] Caught error code, retrying attempt #2
[INFO] running /code/SuperBuild/src/elibs/mve/apps/dmrecon/dmrecon -s3 --progress=fancy --local-neighbors=2 “/datasets/code/mve”
MVE Depth Map Reconstruction (built on Mar 19 2020, 15:23:07)
Initializing scene with 9 views…
View: Unrecognized extension undist-L3.png.new, skipping.
View: Unrecognized extension undist-L3.png.new, skipping.
View: Unrecognized extension undist-L3.png.new, skipping.
View: Unrecognized extension undist-L3.png.new, skipping.
View: Unrecognized extension undist-L3.png.new, skipping.
Initialized 9 views (max ID is 8), took 0ms.
Reading Photosynther file (9 cameras, 1325 features)…
Reconstructing all views…
0 of 9 completed (0.00%)
4 of 9 completed (44.44%)
Reconstruction took 3685ms.
Saving views back to disc…
Saving views to MVE files… done.
[INFO] running /code/SuperBuild/src/elibs/mve/apps/scene2pset/scene2pset -F3 “/datasets/code/mve” “/datasets/code/mve/mve_dense_point_cloud.ply”
MVE Scene to Pointset (built on Mar 19 2020, 15:23:20)
Using depthmap “depth-L3” and color image “undist-L3”
Initializing scene with 9 views…
View: Unrecognized extension undist-L3.png.new, skipping.
View: Unrecognized extension undist-L3.png.new, skipping.
View: Unrecognized extension undist-L3.png.new, skipping.
View: Unrecognized extension undist-L3.png.new, skipping.
View: Unrecognized extension undist-L3.png.new, skipping.
Initialized 9 views (max ID is 8), took 0ms.
Processing view “0003”…
Processing view “0005”…
Processing view “0001”…
Processing view “0004”…
Processing view “0000”…
Processing view “0006” (with colors)…
Processing view “0002” (with colors)…
Processing view “0008” (with colors)…
Processing view “0007” (with colors)…
Writing final point set (0 points)…
Writing PLY file (0 verts, with colors, with normals, with confidences, with values, 0 faces)… done.
[INFO] running /code/SuperBuild/src/elibs/mve/apps/meshclean/meshclean -t0.6 --no-clean --component-size=0 “/datasets/code/mve/mve_dense_point_cloud.ply” “/datasets/code/mve/mve_dense_point_cloud.filtered.ply”
MVE FSSR Mesh Cleaning (built on Mar 19 2020, 15:23:46)
Loading mesh: /datasets/code/mve/mve_dense_point_cloud.ply
PLY Loader: comment Export generated by libmve
Reading PLY: 0 verts… done.
Error: Mesh is empty!
Traceback (most recent call last):
File “/code/run.py”, line 61, in
app.execute()
File “/code/stages/odm_app.py”, line 92, in execute
self.first_stage.run()
File “/code/opendm/types.py”, line 344, in run
self.next_stage.run(outputs)
File “/code/opendm/types.py”, line 344, in run
self.next_stage.run(outputs)
File “/code/opendm/types.py”, line 344, in run
self.next_stage.run(outputs)
File “/code/opendm/types.py”, line 344, in run
self.next_stage.run(outputs)
File “/code/opendm/types.py”, line 325, in run
self.process(self.args, outputs)
File “/code/stages/mve.py”, line 89, in process
system.run(‘%s -t%s --no-clean --component-size=0 “%s” “%s”’ % (context.meshclean_path, min(1.0, args.mve_confidence), tree.mve_model, mve_filtered_model), env_vars={‘OMP_NUM_THREADS’: args.max_concurrency})
File “/code/opendm/system.py”, line 76, in run
raise Exception(“Child returned {}”.format(retcode))
Exception: Child returned 1

5.--min-num-features 12000, (default is 8000)

Last lines of log

/usr/local/lib/python2.7/dist-packages/numpy/core/fromnumeric.py:2920: RuntimeWarning: Mean of empty slice.
out=out, **kwargs)
/usr/local/lib/python2.7/dist-packages/numpy/core/_methods.py:85: RuntimeWarning: invalid value encountered in double_scalars
ret = ret.dtype.type(ret / rcount)
Traceback (most recent call last):
File “/code/SuperBuild/src/opensfm/bin/opensfm”, line 34, in
command.run(args)
File “/code/SuperBuild/src/opensfm/opensfm/commands/reconstruct.py”, line 23, in run
incremental_reconstruction(data, graph)
File “/code/SuperBuild/src/opensfm/opensfm/reconstruction.py”, line 1443, in incremental_reconstruction
data, graph, graph_inliers, reconstruction, remaining_images, camera_priors, gcp)
File “/code/SuperBuild/src/opensfm/opensfm/reconstruction.py”, line 1349, in grow_reconstruction
remove_outliers(graph_inliers, reconstruction, config)
File “/code/SuperBuild/src/opensfm/opensfm/reconstruction.py”, line 1144, in remove_outliers
threshold = get_actual_threshold(config, reconstruction.points)
File “/code/SuperBuild/src/opensfm/opensfm/reconstruction.py”, line 1131, in get_actual_threshold
mean, std = get_error_distribution(points)
File “/code/SuperBuild/src/opensfm/opensfm/reconstruction.py”, line 1122, in get_error_distribution
robust_std = 1.486*np.median(np.linalg.norm(all_errors-robust_mean, axis=1))
File “/usr/local/lib/python2.7/dist-packages/numpy/linalg/linalg.py”, line 2390, in norm
return sqrt(add.reduce(s, axis=axis, keepdims=keepdims))
numpy.core._internal.AxisError: axis 1 is out of bounds for array of dimension 1
Traceback (most recent call last):
File “/code/run.py”, line 61, in
app.execute()
File “/code/stages/odm_app.py”, line 92, in execute
self.first_stage.run()
File “/code/opendm/types.py”, line 344, in run
self.next_stage.run(outputs)
File “/code/opendm/types.py”, line 344, in run
self.next_stage.run(outputs)
File “/code/opendm/types.py”, line 344, in run
self.next_stage.run(outputs)
File “/code/opendm/types.py”, line 325, in run
self.process(self.args, outputs)
File “/code/stages/run_opensfm.py”, line 30, in process
octx.reconstruct(self.rerun())
File “/code/opendm/osfm.py”, line 41, in reconstruct
self.run(‘reconstruct’)
File “/code/opendm/osfm.py”, line 23, in run
(context.opensfm_path, command, self.opensfm_project_path))
File “/code/opendm/system.py”, line 76, in run
raise Exception(“Child returned {}”.format(retcode))
Exception: Child returned 1

I also tried the options below recommended here (thanks @Kostiantyn_Bykhkalo and @pierotofy)

orthophoto-png:true
min-num-features:20000
orthophoto-no-tiled:true
mesh-size:400000
opensfm-depthmap-min-consistent-views:5
merge:orthophoto
texturing-data-term:area
texturing-skip-visibility-test:true
camera-lens perspective

I would greatly appreciate any lead or idea.
Thank you very much!

pierotofy · 7 April 2020 21:43

Is the dataset attached the entire dataset?

If so, perhaps you need more images for a reconstruction to be properly processed.

If you process the green (or red, or blue) band independently do you get a result?

gast · 8 April 2020 06:25

Tnx for the response.
The attached data set is only a sample with 10 images per color, total 30 images. I tried to run with 50 images per color, total of 150, and still got the same error.

When I run each individual color I have no errors. However, running in ‘fast orthophoto’ just to be quick, the orthophotos do not overlap (attached gif). Red and Blue are about 3.7[m] off. Green is off about 58[m] from the other two colors. That is very strange since the coordinates for images of different colors but of the same sequence, i.e *DSC00001.JPG for all colors have the same Lat/Long as below for all files (looking with exiftool before processing with ODM)

[email protected]: ls | grep 00001
blue450_left_DSC00001.JPG
green550_left_DSC00001.JPG
red650_left_DSC00001.JPG

[email protected]: exiftool *00001.JPG | grep Lat
GPS Latitude Ref                : North
GPS Latitude                    : 36 deg 8' 16.20" N
GPS Latitude Ref                : North
GPS Latitude                    : 36 deg 8' 16.20" N
GPS Latitude Ref                : North
GPS Latitude                    : 36 deg 8' 16.20" N

[email protected]: exiftool *00001.JPG | grep Long
GPS Longitude Ref               : West
GPS Longitude                   : 119 deg 0' 45.92" W
GPS Longitude Ref               : West
GPS Longitude                   : 119 deg 0' 45.92" W
GPS Longitude Ref               : West
GPS Longitude                   : 119 deg 0' 45.92" W

myimage

Looking at the output images.json file that was written from each of the three individual runs - for images with serial number DSC00001, all had exactly the same coordinate as below:

[email protected]:~/fast$ perl -e "s/\{/\n/g" -p red/images.json |grep 00001 | awk '{FS=/,/; print $26" "$29" "$30" "$45" "$46}'
"red650_left_DSC00001.JPG", "latitude": 36.13783211111111, "longitude": -119.01275530556886,
 [email protected]:~/fast$ perl -e "s/\{/\n/g" -p green/images.json |grep 00001 | awk '{FS=/,/; print $26" "$29" "$30" "$45" "$46}'
"green550_left_DSC00001.JPG", "latitude": 36.13783211111111, "longitude": -119.01275530556886,
 [email protected]:~/fast$ perl -e "s/\{/\n/g" -p blue/images.json |grep 00001 | awk '{FS=/,/; print $26" "$29" "$30" "$45" "$46}'
"blue450_left_DSC00001.JPG", "latitude": 36.13783211111111, "longitude": -119.01275530556886,
[email protected]:~/fast$

The above is true for all images and coordinates.

I tried one test on one color before and it past. Thanks for the good suggestion to try it on all and look for any issues.

Can this shift cause the Mesh empty error ?

The geo-tag flow is:

Use original RGB image which is not geo-taged.
Split to individual R, G, B images.
Use a GPS location file to geo-tag images where every R,G,B image set - gets the exact geo-tag as shown above.

Lastly - if I overlay the three individual images of step (2), they are perfectly registered as expected.
myimageRGB

link to images.json files from the three runs

pierotofy · 8 April 2020 16:05

I think this is just a difficult dataset to process. Not enough recognizable features, too low of flight elevation, not enough sidelap and not enough scale variation (elevation change). The camera model fails to converge to a reasonable solution and the reconstruction fails.

In short, follow best practices for data capture. https://docs.opendronemap.org/flying.html

gast · 10 April 2020 21:23

Thank you very much for the reference and explanation.

This data is from a mission that was flown at 120 [m] with overlap of more than 70% along (forward) and better than 55% across (sideways) @ GSD of ~2 [cm/pix]. I think these parameters should be compatible with the recommendations unless I am mistaken ?

smathermather · 10 April 2020 23:52

Sidelap should be 60% or (preferably) more (depending on how tall the objects are). 55% is more useful in classical photogrammetry where we have a known position and orientation.

smathermather · 10 April 2020 23:55

Moreover, most multispectral sensors are poor sensors which only have good data near the center of the sensor. This means that we have to collect extra data: some sensors require 80% overlap just to compensate.

gast · 11 April 2020 14:37

Thank you for explanation.
Data here was collected using Sony A6000 which presumably is a better sensor than most other multi-spectral sensors out there as far as I know.

smathermather · 11 April 2020 15:12

Ha! Yes. That is a very fine sensor. Since you appear to be flying over trees, I would bump your overlap above the minimum, and maybe aim for 65%.

gast · 12 April 2020 04:57

Understood.

Actually, at 120[m] with this sensor, the more accurate overlaps are 76% along and 64% across (along swath is 64 [m] and across is 84 [m]. Images taken every 15 [m] along and every 30 [m] across).

I believe there may have been an issue with the sample I used in that it was too small (10 imgs per color only) and it was linear in nature, i.e. 10 imgs per band in a line as subset for a first time sanity test before running hundreds of images. When I used a larger set of roughly 70 images per band that are not linear but are square-tiled, I no longer see the Error: Mesh is empty !. However, I still do get problems with reconstruction as seen in log below.
I understand the arguments regarding number of recognizable features and elevation change. Yet, it still seems that these mission parameters are not that far off if at all according to the published recommendations.

Last lines of log

/usr/local/lib/python2.7/dist-packages/numpy/core/fromnumeric.py:2920: RuntimeWarning: Mean of empty slice.
out=out, **kwargs)
/usr/local/lib/python2.7/dist-packages/numpy/core/_methods.py:85: RuntimeWarning: invalid value encountered in double_scalars
ret = ret.dtype.type(ret / rcount)
Traceback (most recent call last):
File "/code/SuperBuild/src/opensfm/bin/opensfm", line 34, in <module>
command.run(args)
File "/code/SuperBuild/src/opensfm/opensfm/commands/reconstruct.py", line 23, in run
incremental_reconstruction(data, graph)
File "/code/SuperBuild/src/opensfm/opensfm/reconstruction.py", line 1443, in incremental_reconstruction
data, graph, graph_inliers, reconstruction, remaining_images, camera_priors, gcp)
File "/code/SuperBuild/src/opensfm/opensfm/reconstruction.py", line 1349, in grow_reconstruction
remove_outliers(graph_inliers, reconstruction, config)
File "/code/SuperBuild/src/opensfm/opensfm/reconstruction.py", line 1144, in remove_outliers
threshold = get_actual_threshold(config, reconstruction.points)
File "/code/SuperBuild/src/opensfm/opensfm/reconstruction.py", line 1131, in get_actual_threshold
mean, std = get_error_distribution(points)
File "/code/SuperBuild/src/opensfm/opensfm/reconstruction.py", line 1122, in get_error_distribution
robust_std = 1.486*np.median(np.linalg.norm(all_errors-robust_mean, axis=1))
File "/usr/local/lib/python2.7/dist-packages/numpy/linalg/linalg.py", line 2390, in norm
return sqrt(add.reduce(s, axis=axis, keepdims=keepdims))
numpy.core._internal.AxisError: axis 1 is out of bounds for array of dimension 1
Traceback (most recent call last):
File "/code/run.py", line 61, in <module>
app.execute()
File "/code/stages/odm_app.py", line 92, in execute
self.first_stage.run()
File "/code/opendm/types.py", line 344, in run
self.next_stage.run(outputs)
File "/code/opendm/types.py", line 344, in run
self.next_stage.run(outputs)
File "/code/opendm/types.py", line 344, in run
self.next_stage.run(outputs)
File "/code/opendm/types.py", line 325, in run
self.process(self.args, outputs)
File "/code/stages/run_opensfm.py", line 32, in process
octx.reconstruct(self.rerun())
File "/code/opendm/osfm.py", line 41, in reconstruct
self.run('reconstruct')
File "/code/opendm/osfm.py", line 23, in run
(context.opensfm_path, command, self.opensfm_project_path))
File "/code/opendm/system.py", line 76, in run
raise Exception("Child returned {}".format(retcode))
Exception: Child returned 1

I am attaching the larger set of roughly 70 images per band . If anyone has any clues as to the reconstruction issues, I’d be love to hear.

https://drive.google.com/file/d/17uwHVWg0jY71G__plVpkKwHitpbzKyOH/view?usp=sharing

ODM is a great product and open source effort. which I’d love to use it. I still think data should be ok and there may be something else I am missing in how to use optimally ODM to get it to work.

Cheers!

pierotofy · 15 April 2020 02:29

Here’s what I don’t understand about this camera… if the bands are all captured from the same lens, why are they saved as separate JPEGs? Why not just combine the JPEGs into a single file with 3 bands (RGB) and process it as a single camera setup?

gast · 15 April 2020 11:08

Good question.

My experience with multi-spectral on pix4D is that many times, some individual channels that are spectrally limited by the entrance filter (20nm or so) may have less information and features than others and hence the output ortho (which is solved individually in a single camera setup) will not be as good as when all channels (individually spectrally limited) are solved as a multi-camera rig. Many times green channels have more information than blue or NIR etc and so it ends up that “stronger” channels seem to compensate for the “weaker” channels.

Also - if I have say 6 channels in two different JPGs (or 12 in 4 JPGs) I’ll have to run twice (or 4 times) instead of one multi-camera run. So in terms of throughput it may not be beneficial. That is still to be shown though. It may be the case that 2 x single camera runs would not take longer than a single multi-camera run.

pierotofy · 15 April 2020 12:57

In terms of throughput, if speed is a concern, your assumption might not be correct; if you have 5 bands and 10 shots, you need to process a dataset with 5 * 10 = 50 images, but if you have 2 sets of 3-channel images and 10 shots, you only need to run the pipeline twice with 2 * 10 = 20 images, which I bet will actually decrease runtime… (you might have the usual alignment issues though).

Processing the full dataset with merged bands worked out pretty well.

This camera is quite unique, I think the multi-camera setup in ODM is not well suited for it at the moment, not without making changes. The fact that a single lens captures the images at different bands should be used to a) Process a single band to solve the SFM problem - and - b) Run texturing using the camera poses computed from a single band to process all bands. This would require some substantial changes to pull off, but it’s possible…

In the meanwhile, I would stick to merging bands together as a workaround.

gast · 15 April 2020 16:58

You’re correct about the alignment which I didn’t touch on at the previous post. The outputs will not be aligned if running in merged-bands and the mosaics would need to be aligned afterwards.

Your point about the TPT is well taken. I would love to compare these, however I was not able to run in “MicaSense” mode yet…

Were you successful in running this set using single bands rather than merged? Should there be any fundamental reason why it would not work in ODM ?

pierotofy · 15 April 2020 17:24

Photogrammetry needs motion; two or more images that look almost identical and have no relative motion will trip up the reconstruction (which is the case with this camera). So no.

I’d be curious to see results from other software (exported as an N-band ortho).

It’s quite possible that other software is being smarter about identifying images coming from the same lens and modifying the reconstruction workflow.

gast · 16 April 2020 09:28

Meant to say relative motion between cameras (and not between successive triggers) ? Where in mica case for instance they are physically not co-located to some degree, as opposed to agrowing where several bands utilize the same lens?

Attached orthos from pix4D. Output can be optimized.
https://drive.google.com/file/d/1Gu4MN8otdki-FlafpVz2Hkq7IcymGBqG/view?usp=sharing

pierotofy · 16 April 2020 13:51

Thanks for the files; like I mentioned, seems like Pix4D is being smarter.

I don’t have a solution to this for the time being, except the workaround to merge your band files prior to processing and process it as a 3-band input.

This is something that needs improvement. Pull requests welcome.

gast · 16 April 2020 14:13

I’ll try the work around.
I was able to solve the 3-band single camera input. What I hope will work is to solve say two or four cameras that are each 3-band input, where there is a good angular difference between the center of each ‘camera’ or part of sensor that acts as a camera. Then hopefully there should be enough relative motion between the cameras for reconstruction.