Cannot Process Dataset Error

[mspiess]

I’m new to WebODM. I have collected images for 65ac. (260 RGB images). 80% front overlap and 70% side. I have tried various settings (fast ortho, high resolution) and the job fails with the error. I have been able to process some smaller datasets including a sub set of the one I’m trying to process. Suggestions?
Thanks!

System: W10, 8GB RAM, 120GB free disk
Data: Montair.zip - Google Drive
Output:

[INFO] Finished odm_filterpoints stage
[INFO] Running odm_meshing stage
[INFO] Writing ODM 2.5D Mesh file in: C:\WebODM\resources\app\apps\NodeODM\data\753ed0e7-07c9-4a2b-b031-9affb0f3d737\odm_meshing\odm_25dmesh.ply
[INFO] ODM 2.5D DSM resolution: 0.22785000000000002
[INFO] Created temporary directory: C:\WebODM\resources\app\apps\NodeODM\data\753ed0e7-07c9-4a2b-b031-9affb0f3d737\odm_meshing\tmp
[INFO] Creating DSM for 2.5D mesh
[INFO] running pdal info “C:\WebODM\resources\app\apps\NodeODM\data\753ed0e7-07c9-4a2b-b031-9affb0f3d737\odm_filterpoints\point_cloud.ply” > “C:\Users\mspiess\AppData\Local\Temp\tmpnvqea9vh.json”
[INFO] Point cloud bounds are [minx: -335.0365295, maxx: 543.3630371] [miny: -460.828186, maxy: 311.4981079]
[INFO] DEM resolution is (3390, 3856), max tile size is 4096, will split DEM generation into 1 tiles
[INFO] Generating C:\WebODM\resources\app\apps\NodeODM\data\753ed0e7-07c9-4a2b-b031-9affb0f3d737\odm_meshing\tmp\mesh_dsm_r0.6444571203734195_x0_y0.tif (max, radius: 0.6444571203734195, resolution: 0.22785000000000002)
[INFO] running pdal pipeline -i C:\Users\mspiess\AppData\Local\Temp\tmp248kpiv8.json
[INFO] running gdalbuildvrt -input_file_list “C:\WebODM\resources\app\apps\NodeODM\data\753ed0e7-07c9-4a2b-b031-9affb0f3d737\odm_meshing\tmp\tiles_list.txt” “C:\WebODM\resources\app\apps\NodeODM\data\753ed0e7-07c9-4a2b-b031-9affb0f3d737\odm_meshing\tmp\tiles.vrt”
0…10…20…30…40…50…60…70…80…90…100 - done.
[INFO] running gdal_translate -co NUM_THREADS=4 -co BIGTIFF=IF_SAFER --config GDAL_CACHEMAX 22.4% “C:\WebODM\resources\app\apps\NodeODM\data\753ed0e7-07c9-4a2b-b031-9affb0f3d737\odm_meshing\tmp\tiles.vrt” “C:\WebODM\resources\app\apps\NodeODM\data\753ed0e7-07c9-4a2b-b031-9affb0f3d737\odm_meshing\tmp\tiles.tmp.tif”
Input file size is 3856, 3390
0…10…20…30…40…50…60…70…80…90…100 - done.
[INFO] running gdal_translate -co NUM_THREADS=4 -co BIGTIFF=IF_SAFER --config GDAL_CACHEMAX 22.4% -outsize 10% 0 “C:\WebODM\resources\app\apps\NodeODM\data\753ed0e7-07c9-4a2b-b031-9affb0f3d737\odm_meshing\tmp\tiles.tmp.tif” “C:\WebODM\resources\app\apps\NodeODM\data\753ed0e7-07c9-4a2b-b031-9affb0f3d737\odm_meshing\tmp\tiles.small.tif”
0
.
.
.
10
.
.
.
20
.
.
.
30
.
.
.
40
.
.
.
50
.
.
.
60
.
.
.
70
.
.
.
80
.
.
.
90
.
.
.
Input file size is 3856, 3390
0…10…20…30…40…50…60…70…80…90…100 - done.
[INFO] running gdalbuildvrt -resolution highest -r bilinear “C:\WebODM\resources\app\apps\NodeODM\data\753ed0e7-07c9-4a2b-b031-9affb0f3d737\odm_meshing\tmp\merged.vrt” “C:\WebODM\resources\app\apps\NodeODM\data\753ed0e7-07c9-4a2b-b031-9affb0f3d737\odm_meshing\tmp\tiles.small_filled.tif” “C:\WebODM\resources\app\apps\NodeODM\data\753ed0e7-07c9-4a2b-b031-9affb0f3d737\odm_meshing\tmp\tiles.tmp.tif”
0…10…20…30…40…50…60…70…80…90…100 - done.
[INFO] running gdal_translate -co NUM_THREADS=4 -co TILED=YES -co BIGTIFF=IF_SAFER -co COMPRESS=DEFLATE --config GDAL_CACHEMAX 22.4% “C:\WebODM\resources\app\apps\NodeODM\data\753ed0e7-07c9-4a2b-b031-9affb0f3d737\odm_meshing\tmp\merged.vrt” “C:\WebODM\resources\app\apps\NodeODM\data\753ed0e7-07c9-4a2b-b031-9affb0f3d737\odm_meshing\tmp\tiles.tif”
Input file size is 3856, 3390
0…10…20…30…40…50…60…70…80…90…100 - done.
[INFO] Completed mesh_dsm.tif in 0:00:05.688173
[INFO] Sampling points from DSM: C:\WebODM\resources\app\apps\NodeODM\data\753ed0e7-07c9-4a2b-b031-9affb0f3d737\odm_meshing\tmp\mesh_dsm.tif
[INFO] running “C:\WebODM\resources\app\apps\ODM\SuperBuild\install\bin\dem2points” -inputFile “C:\WebODM\resources\app\apps\NodeODM\data\753ed0e7-07c9-4a2b-b031-9affb0f3d737\odm_meshing\tmp\mesh_dsm.tif” -outputFile “C:\WebODM\resources\app\apps\NodeODM\data\753ed0e7-07c9-4a2b-b031-9affb0f3d737\odm_meshing\tmp\dsm_points.ply” -skirtHeightThreshold 1.5 -skirtIncrements 0.2 -skirtHeightCap 100
[INFO] running “C:\WebODM\resources\app\apps\ODM\SuperBuild\install\bin\PoissonRecon” --in “C:\WebODM\resources\app\apps\NodeODM\data\753ed0e7-07c9-4a2b-b031-9affb0f3d737\odm_meshing\tmp\dsm_points.ply” --out “C:\WebODM\resources\app\apps\NodeODM\data\753ed0e7-07c9-4a2b-b031-9affb0f3d737\odm_meshing\odm_25dmesh.dirty.ply” --depth 11 --pointWeight 4 --samplesPerNode 1.0 --threads 3 --maxMemory 2 --bType 2 --linearFit
[WARNING] Child returned 3221225477
[WARNING] PoissonRecon failed with 1 threads, let’s retry with 0…
[INFO] running “C:\WebODM\resources\app\apps\ODM\SuperBuild\install\bin\PoissonRecon” --in “C:\WebODM\resources\app\apps\NodeODM\data\753ed0e7-07c9-4a2b-b031-9affb0f3d737\odm_meshing\tmp\dsm_points.ply” --out “C:\WebODM\resources\app\apps\NodeODM\data\753ed0e7-07c9-4a2b-b031-9affb0f3d737\odm_meshing\odm_25dmesh.dirty.ply” --depth 11 --pointWeight 4 --samplesPerNode 1.0 --threads 1 --maxMemory 2 --bType 2 --linearFit
[WARNING] Child returned 3221226505
[INFO] running “C:\WebODM\resources\app\apps\ODM\SuperBuild\install\bin\OpenMVS\ReconstructMesh” -i “C:\WebODM\resources\app\apps\NodeODM\data\753ed0e7-07c9-4a2b-b031-9affb0f3d737\odm_meshing\odm_25dmesh.dirty.ply” -o “C:\WebODM\resources\app\apps\NodeODM\data\753ed0e7-07c9-4a2b-b031-9affb0f3d737\odm_meshing\odm_25dmesh.ply” --remove-spikes 0 --remove-spurious 20 --smooth 0 --target-face-num 400000
===== Dumping Info for Geeks (developers need this to fix bugs) =====
Child returned 1
Traceback (most recent call last):
File “C:\WebODM\resources\app\apps\ODM\stages\odm_app.py”, line 89, in execute
self.first_stage.run()
File “C:\WebODM\resources\app\apps\ODM\opendm\types.py”, line 340, in run
self.next_stage.run(outputs)
File “C:\WebODM\resources\app\apps\ODM\opendm\types.py”, line 340, in run
self.next_stage.run(outputs)
File “C:\WebODM\resources\app\apps\ODM\opendm\types.py”, line 340, in run
self.next_stage.run(outputs)
[Previous line repeated 2 more times]
File “C:\WebODM\resources\app\apps\ODM\opendm\types.py”, line 321, in run
self.process(self.args, outputs)
File “C:\WebODM\resources\app\apps\ODM\stages\odm_meshing.py”, line 66, in process
mesh.create_25dmesh(tree.filtered_point_cloud, tree.odm_25dmesh,
File “C:\WebODM\resources\app\apps\ODM\opendm\mesh.py”, line 43, in create_25dmesh
mesh = screened_poisson_reconstruction(dsm_points, outMesh, depth=depth,
File “C:\WebODM\resources\app\apps\ODM\opendm\mesh.py”, line 207, in screened_poisson_reconstruction
system.run(’"{reconstructmesh}" -i “{infile}” ’
File “C:\WebODM\resources\app\apps\ODM\opendm\system.py”, line 106, in run
raise SubprocessException(“Child returned {}”.format(retcode), retcode)
opendm.system.SubprocessException: Child returned 1

===== Done, human-readable information to follow… =====

[ERROR] Uh oh! Processing stopped because of strange values in the reconstruction. This is often a sign that the input data has some issues or the software cannot deal with it. Have you followed best practices for data acquisition? See https://docs.opendronemap.org/flying.html
100 - done.

[Saijin_Naib]

Welcome!

Sorry that you’re having trouble getting this dataset to reconstruct properly.

I’m giving it a test now:

[Saijin_Naib]

Processing Parameters:

Options: cog: true, crop: 0, debug: true, dem-resolution: 1, dsm: true, matcher-distance: 0, matcher-neighbors: 0, min-num-features: 12000, orthophoto-resolution: 1, pc-geometric: true, pc-quality: high, use-3dmesh: true, verbose: true

Orthophoto:

DSM:

Report:
mspiess_Report.pdf (9.5 MB)

---

I did some pre-processing (Optional):

---

Your likely issue:
8GB RAM is not sufficient for this size dataset, unless you very purposefully reduce quality parameters, and even then, I’d say it might be iffy.

---

Download Link for All Products (will be deleted shortly):
https://wln1.nyc3.cdn.digitaloceanspaces.com/65d3091f-e952-4276-9992-560eb1dbeee6/all.zip

[manand881]

can you please elaborate on the kind of preprocessing that you did? and touch upon why you chose to do it, its advantages, its necessity and benefits?

[Saijin_Naib]

Sure.

The pre-processing was done with XNConvert. The settings are pictured above.

It was for aesthetic reasons with this dataset (makes orthophoto and DSM look really nice, IMO), but on other datasets that fail to reconstruct it can help to increase tie point extraction.

Feature Request:

Feature Request: Perform Edge Enhancement and Histogram Adjustment for Matching only

[mspiess]

To answer previous post… Yes the 8GB was probably a stretch. I was not maxing out, but close. I will order some more and try again. I was able to process the data set using the online node. My interest is primarily in orthophotos at this point for use in agriculture. This was RGB data, but I will use Near-IR as well. I have been working on curriculum development for agriculture teachers and for ways to keep costs down as they are non-commercial users and rarely have budgets to cover commercial products like DroneDeploy. Thanks for your assistance.
Mike

[Saijin_Naib]

Lightning is very capable, yes.

May I remove the processed data linked above?

NIR imagery will work great provided you have proper exposure.

You can try these NIR datasets if you wish:

GitHub - Saijin-Naib/sUAS_Photogrammetry_Suite_Test_Data: Data tested against...

[mspiess]

Yes I have the data, thanks.