Task fail on tower reconstruction at meshing step — OOM error

Gordon · 13 May 2022 01:34

Attempting to do a detailed 3D model of a power pole (which has lots of features due to it falling apart!) I encountered this PoissonRecon failure.
Any suggestions as to what might cause that?

323 images 60:14:11

Cannot process dataset

|Options:|auto-boundary: true, boundary: {“type”:“FeatureCollection”,“features”:[{“type”:“Feature”,“properties”:{},“geometry”:{“type”:“Polygon”,“coordinates”:[[[151.05253279209137,-31.331298169293767]…,]}, dem-resolution: 1, dsm: true, dtm: true, feature-quality: ultra, gps-accuracy: 6, mesh-octree-depth: 13, mesh-size: 350000, orthophoto-resolution: 0.3, pc-filter: 5, pc-geometric: true, pc-quality: ultra, resize-to: -1, use-3dmesh: true|

End of the console log -

?> Parameters:
input = E:\WebODM\resources\app\apps\NodeODM\data\981e80b5-02b5-4c9c-ba9b-384caef99f1d\opensfm\undistorted\openmvs\scene_dense_dense_filtered.ply
output = E:\WebODM\resources\app\apps\NodeODM\data\981e80b5-02b5-4c9c-ba9b-384caef99f1d\odm_filterpoints\point_cloud.ply
std = 5
meanK = 16
boundary = 17 polygon vertexes
concurrency = 16
verbose = no

→ Setting num_threads to 16

→ Cropping

→ Done cropping in 104.8s

?> Skipping sampling

→ Statistical filtering

?> Done in 288s

→ Writing output

?> Done in 170.4s

?> Pipeline done in 563.2s

[INFO] Finished odm_filterpoints stage
[INFO] Running odm_meshing stage
[INFO] Writing ODM Mesh file in: E:\WebODM\resources\app\apps\NodeODM\data\981e80b5-02b5-4c9c-ba9b-384caef99f1d\odm_meshing\odm_mesh.ply
[INFO] running “E:\WebODM\resources\app\apps\ODM\SuperBuild\install\bin\PoissonRecon” --in “E:\WebODM\resources\app\apps\NodeODM\data\981e80b5-02b5-4c9c-ba9b-384caef99f1d\odm_filterpoints\point_cloud.ply” --out “E:\WebODM\resources\app\apps\NodeODM\data\981e80b5-02b5-4c9c-ba9b-384caef99f1d\odm_meshing\odm_mesh.dirty.ply” --depth 13 --pointWeight 4.0 --samplesPerNode 1.0 --threads 15 --bType 2 --linearFit
[WARNING] Child returned 3221226505
[WARNING] PoissonRecon failed with 7 threads, let’s retry with 3…
[INFO] running “E:\WebODM\resources\app\apps\ODM\SuperBuild\install\bin\PoissonRecon” --in “E:\WebODM\resources\app\apps\NodeODM\data\981e80b5-02b5-4c9c-ba9b-384caef99f1d\odm_filterpoints\point_cloud.ply” --out “E:\WebODM\resources\app\apps\NodeODM\data\981e80b5-02b5-4c9c-ba9b-384caef99f1d\odm_meshing\odm_mesh.dirty.ply” --depth 13 --pointWeight 4.0 --samplesPerNode 1.0 --threads 7 --bType 2 --linearFit
[WARNING] Child returned 3221226505
[WARNING] PoissonRecon failed with 3 threads, let’s retry with 1…
[INFO] running “E:\WebODM\resources\app\apps\ODM\SuperBuild\install\bin\PoissonRecon” --in “E:\WebODM\resources\app\apps\NodeODM\data\981e80b5-02b5-4c9c-ba9b-384caef99f1d\odm_filterpoints\point_cloud.ply” --out “E:\WebODM\resources\app\apps\NodeODM\data\981e80b5-02b5-4c9c-ba9b-384caef99f1d\odm_meshing\odm_mesh.dirty.ply” --depth 13 --pointWeight 4.0 --samplesPerNode 1.0 --threads 3 --bType 2 --linearFit
[WARNING] Child returned 3221226505
[WARNING] PoissonRecon failed with 1 threads, let’s retry with 0…
[INFO] running “E:\WebODM\resources\app\apps\ODM\SuperBuild\install\bin\PoissonRecon” --in “E:\WebODM\resources\app\apps\NodeODM\data\981e80b5-02b5-4c9c-ba9b-384caef99f1d\odm_filterpoints\point_cloud.ply” --out “E:\WebODM\resources\app\apps\NodeODM\data\981e80b5-02b5-4c9c-ba9b-384caef99f1d\odm_meshing\odm_mesh.dirty.ply” --depth 13 --pointWeight 4.0 --samplesPerNode 1.0 --threads 1 --bType 2 --linearFit
[WARNING] Child returned 3221226505
[INFO] running “E:\WebODM\resources\app\apps\ODM\SuperBuild\install\bin\OpenMVS\ReconstructMesh” -i “E:\WebODM\resources\app\apps\NodeODM\data\981e80b5-02b5-4c9c-ba9b-384caef99f1d\odm_meshing\odm_mesh.dirty.ply” -o “E:\WebODM\resources\app\apps\NodeODM\data\981e80b5-02b5-4c9c-ba9b-384caef99f1d\odm_meshing\odm_mesh.ply” --remove-spikes 0 --remove-spurious 20 --smooth 0 --target-face-num 700000 -v 0
===== Dumping Info for Geeks (developers need this to fix bugs) =====
Child returned 1
Traceback (most recent call last):
File “E:\WebODM\resources\app\apps\ODM\stages\odm_app.py”, line 94, in execute
self.first_stage.run()
File “E:\WebODM\resources\app\apps\ODM\opendm\types.py”, line 346, in run
self.next_stage.run(outputs)
File “E:\WebODM\resources\app\apps\ODM\opendm\types.py”, line 346, in run
self.next_stage.run(outputs)
File “E:\WebODM\resources\app\apps\ODM\opendm\types.py”, line 346, in run
self.next_stage.run(outputs)
[Previous line repeated 3 more times]
File “E:\WebODM\resources\app\apps\ODM\opendm\types.py”, line 327, in run
self.process(self.args, outputs)
File “E:\WebODM\resources\app\apps\ODM\stages\odm_meshing.py”, line 24, in process
mesh.screened_poisson_reconstruction(tree.filtered_point_cloud,
File “E:\WebODM\resources\app\apps\ODM\opendm\mesh.py”, line 208, in screened_poisson_reconstruction
system.run(’"{reconstructmesh}" -i “{infile}” ’
File “E:\WebODM\resources\app\apps\ODM\opendm\system.py”, line 106, in run
raise SubprocessException(“Child returned {}”.format(retcode), retcode)
opendm.system.SubprocessException: Child returned 1

===== Done, human-readable information to follow… =====

[ERROR] Uh oh! Processing stopped because of strange values in the reconstruction. This is often a sign that the input data has some issues or the software cannot deal with it. Have you followed best practices for data acquisition? See Flying Tips — OpenDroneMap 2.8.4 documentation

smathermather · 13 May 2022 01:47

Turn in auto-boundary. You’re reconstructing things far from the tower and running out of RAM.

Gordon · 13 May 2022 01:59

Plus I had a boundary set (which has worked for lower setting reconstructions), so auto-boundary was ignored

[WARNING] --auto-boundary set but so is --boundary, will use --boundary

Running performance monitor, which never showed more than about 80% of RAM + Virtual in use

smathermather · 13 May 2022 02:05

Oh, I thought I read it carefully… . Towers are tricky, did you do any background removal? rembg will do the trick: it will destroy some of the images, but probably keep enough details of the tower to get a good reconstruction. I’ve had lots of luck doing that.

You could also try turning down your mesh-octree-depth to 12, but I’m guessing you don’t want to .

smathermather · 13 May 2022 02:11

As it occurs, towers are harder than pine cones.

Gordon · 13 May 2022 02:25

I had used mesh-octree-depth 12 on earlier attempts, but there were a few non-real features, and some of the background was stuck to pole features

In Meshlab-

and

smathermather · 13 May 2022 02:59

This confirms my suspicion that it’s a memory error at octree depth of 13, which means either preprocessing to remove background or turn down the octree depth.

Preprocessing is fun! And it does some really compelling things with datasets like this.

Gordon · 15 May 2022 07:53

Reducing pc_quality to high allowed the process to complete with an earlier slightly smaller set of images.

285 images 24:59:35

|Processing Node:|node-odm-1 (auto)|
|Options:|auto-boundary: true, boundary: {“type”:“FeatureCollection”,“features”:[{“type”:“Feature”,“properties”:{},“geometry”:{“type”:“Polygon”,“coordinates”:[[[151.05253279209137,-31.331298169293767],…]]]}}]}, crop: 5, feature-quality: ultra, gps-accuracy: 6, mesh-octree-depth: 13, mesh-size: 350000, pc-filter: 5, pc-geometric: true, pc-quality: high, use-3dmesh: true|
|Average GSD:|0.22 cm|
|Area:|176.69 m²|
|Reconstructed Points:|25,034,236|

The top of the pole is missing, but that is ok, as there is nothing of interest there.

A piece splitting off the side here

Looking up the pole at the space where a large piece has already fallen off.

The line carries 3phase 33kV across the front of my place, so I hope they replace it before it falls over!

Saijin_Naib · 15 May 2022 15:43

Just astounding stuff, really!