Holes in my clouds (no silver lining)

[alanterra]

I have now had three stitchings fail at “ultra” quality (while other runs with ultra with different photos have succeeded). The symptoms are that I have “holes” in my point cloud and exactly corresponding flat areas in my dsm & dtm. If I had to guess, I would say the WebODM exceeded some limit on resources available, and stopped processing for a while, but didn’t crash.

This is clearly not due to a lack of features, as the point cloud and dsm does not have holes when processed at “high” quality. Any suggestions?

This is the latest example. 347 images each 5472 pixels wide, high overlap (90% I think), processed with Docker having 64 GB of memory and 11 processors. (MacOS Intel).

Here is a link to a screenshot from CloudCompare of the .laz file:

Here is a link to a screenshot from QGIS of the dsm.tif file:

Here is a link to the entire output from the run:

task_output.txt

And these are the parameters of the run:

[INFO] DTM is turned on, automatically turning on point cloud classification
[INFO] Initializing ODM 3.0.4 - Sun Feb 05 07:21:44 2023
[INFO] ==============
[INFO] 3d_tiles: False
[INFO] align: None
[INFO] auto_boundary: True
[INFO] auto_boundary_distance: 0
[INFO] bg_removal: False
[INFO] boundary: {}
[INFO] build_overviews: False
[INFO] camera_lens: auto
[INFO] cameras: {‘dji fc3411 5472 3648 brown 0.6111’: {‘projection_type’: ‘brown’, ‘width’: 5472, ‘height’: 3648, ‘focal_x’: 0.6283419967936914, ‘focal_y’: 0.6283419967936914, ‘c_x’: 0.007569731737564364, ‘c_y’: 0.002472184071427966, ‘k1’: -0.06882044698571836, ‘k2’: 0.04194169673011737, ‘p1’: 0.00040774105953342163, ‘p2’: 0.00037267705432290807, ‘k3’: 0.018507473910662615}}
[INFO] cog: True
[INFO] copy_to: None
[INFO] crop: 3
[INFO] dem_decimation: 1
[INFO] dem_euclidean_map: False
[INFO] dem_gapfill_steps: 3
[INFO] dem_resolution: 2.0
[INFO] dsm: True
[INFO] dtm: True
[INFO] end_with: odm_postprocess
[INFO] fast_orthophoto: False
[INFO] feature_quality: high
[INFO] feature_type: sift
[INFO] force_gps: False
[INFO] gcp: None
[INFO] geo: None
[INFO] gltf: True
[INFO] gps_accuracy: 10
[INFO] ignore_gsd: False
[INFO] matcher_neighbors: 0
[INFO] matcher_type: flann
[INFO] max_concurrency: 11
[INFO] merge: all
[INFO] mesh_octree_depth: 11
[INFO] mesh_size: 200000
[INFO] min_num_features: 10000
[INFO] name: 1f6a9151-7bbf-472d-ba8c-d1bbd23546d1
[INFO] no_gpu: False
[INFO] optimize_disk_space: False
[INFO] orthophoto_compression: DEFLATE
[INFO] orthophoto_cutline: False
[INFO] orthophoto_kmz: False
[INFO] orthophoto_no_tiled: False
[INFO] orthophoto_png: False
[INFO] orthophoto_resolution: 1.0
[INFO] pc_classify: True
[INFO] pc_copc: False
[INFO] pc_csv: False
[INFO] pc_ept: True
[INFO] pc_filter: 2.5
[INFO] pc_las: False
[INFO] pc_quality: ultra
[INFO] pc_rectify: False
[INFO] pc_sample: 0
[INFO] pc_skip_geometric: False
[INFO] pc_tile: False
[INFO] primary_band: auto
[INFO] project_path: /var/www/data
[INFO] radiometric_calibration: none
[INFO] rerun: None
[INFO] rerun_all: False
[INFO] rerun_from: None
[INFO] rolling_shutter: False
[INFO] rolling_shutter_readout: 0
[INFO] sfm_algorithm: incremental
[INFO] skip_3dmodel: False
[INFO] skip_band_alignment: False
[INFO] skip_orthophoto: False
[INFO] skip_report: False
[INFO] sky_removal: False
[INFO] sm_cluster: None
[INFO] sm_no_align: False
[INFO] smrf_scalar: 1.25
[INFO] smrf_slope: 0.15
[INFO] smrf_threshold: 0.5
[INFO] smrf_window: 18.0
[INFO] split: 999999
[INFO] split_image_groups: None
[INFO] split_overlap: 150
[INFO] texturing_keep_unseen_faces: False
[INFO] texturing_single_material: False
[INFO] texturing_skip_global_seam_leveling: False
[INFO] texturing_skip_local_seam_leveling: False
[INFO] tiles: False
[INFO] use_3dmesh: False
[INFO] use_exif: False
[INFO] use_fixed_camera_params: False
[INFO] use_hybrid_bundle_adjustment: False
[INFO] video_limit: 500
[INFO] video_resolution: 4000
[INFO] ==============
[INFO] Running dataset stage
[INFO] Loading dataset from: /var/www/data/1f6a9151-7bbf-472d-ba8c-d1bbd23546d1/images
[INFO] Loading 347 images
[INFO] Wrote images database: /var/www/data/1f6a9151-7bbf-472d-ba8c-d1bbd23546d1/images.json
[INFO] Found 347 usable images
[INFO] Finished dataset stage
[INFO] Running split stage
[INFO] Normal dataset, will process all at once.
[INFO] Finished split stage
[INFO] Running merge stage
[INFO] Normal dataset, nothing to merge.
[INFO] Finished merge stage
[INFO] Running opensfm stage
[INFO] Wrote camera_models_overrides.json to OpenSfM directory
[INFO] Maximum photo dimensions: 5472px
[INFO] Photo dimensions for feature extraction: 2736px
[INFO] nvidia-smi not found in PATH, using CPU
[INFO] Altitude data detected, enabling it for GPS alignment
[INFO] [‘use_exif_size: no’, ‘flann_algorithm: KDTREE’, ‘feature_process_size: 2736’, ‘feature_min_frames: 10000’, ‘processes: 11’, ‘matching_gps_neighbors: 0’, ‘matching_gps_distance: 0’, ‘matching_graph_rounds: 50’, ‘optimize_camera_parameters: yes’, ‘reconstruction_algorithm: incremental’, ‘undistorted_image_format: tif’, ‘bundle_outlier_filtering_type: AUTO’, ‘sift_peak_threshold: 0.066’, ‘align_orientation_prior: vertical’, ‘triangulation_type: ROBUST’, ‘retriangulation_ratio: 2’, ‘matcher_type: FLANN’, ‘feature_type: SIFT’, ‘use_altitude_tag: yes’, ‘align_method: auto’, ‘local_bundle_radius: 0’]
[INFO] Wrote reference_lla.json
[INFO] running “/code/SuperBuild/install/bin/opensfm/bin/opensfm” detect_features “/var/www/data/1f6a9151-7bbf-472d-ba8c-d1bbd23546d1/opensfm”

[Saijin_Naib]

Have you been able to run with feature-quality ultra, more min-num-features (try 64000), pc-filter 0, and pc-classify 0?

[alanterra]

I ran it as you suggested, Saijin, and now there is a different hole. Of course, the hole is in exactly the part of the point cloud of interest to me. As you can see, there is no correspondence between the part that failed to generate on the original run, and the part from the run with the parameters you suggested.

The point cloud is really too large to be useful, and I’m not sure it is worthwhile for me to pursue this, but I will say it is mighty frustrating to do a long (1.5 day) run only to have it silently fail.

In case you want to see them, these are the parameters from this second run:

[INFO] Initializing ODM 3.0.4 - Tue Feb 07 18:46:30 2023
[INFO] ==============
[INFO] 3d_tiles: False
[INFO] align: None
[INFO] auto_boundary: True
[INFO] auto_boundary_distance: 0
[INFO] bg_removal: False
[INFO] boundary: {}
[INFO] build_overviews: False
[INFO] camera_lens: auto
[INFO] cameras: {‘dji fc3411 5472 3648 brown 0.6111’: {‘projection_type’: ‘brown’, ‘width’: 5472, ‘height’: 3648, ‘focal_x’: 0.6283419967936914, ‘focal_y’: 0.6283419967936914, ‘c_x’: 0.007569731737564364, ‘c_y’: 0.002472184071427966, ‘k1’: -0.06882044698571836, ‘k2’: 0.04194169673011737, ‘p1’: 0.00040774105953342163, ‘p2’: 0.00037267705432290807, ‘k3’: 0.018507473910662615}}
[INFO] cog: True
[INFO] copy_to: None
[INFO] crop: 3
[INFO] dem_decimation: 1
[INFO] dem_euclidean_map: False
[INFO] dem_gapfill_steps: 3
[INFO] dem_resolution: 2.0
[INFO] dsm: True
[INFO] dtm: False
[INFO] end_with: odm_postprocess
[INFO] fast_orthophoto: False
[INFO] feature_quality: high
[INFO] feature_type: sift
[INFO] force_gps: False
[INFO] gcp: None
[INFO] geo: None
[INFO] gltf: True
[INFO] gps_accuracy: 10
[INFO] ignore_gsd: False
[INFO] matcher_neighbors: 0
[INFO] matcher_type: flann
[INFO] max_concurrency: 11
[INFO] merge: all
[INFO] mesh_octree_depth: 11
[INFO] mesh_size: 200000
[INFO] min_num_features: 64000
[INFO] name: 70aea463-22a8-4c94-85ee-81ae8876fd40
[INFO] no_gpu: False
[INFO] optimize_disk_space: False
[INFO] orthophoto_compression: DEFLATE
[INFO] orthophoto_cutline: False
[INFO] orthophoto_kmz: False
[INFO] orthophoto_no_tiled: False
[INFO] orthophoto_png: False
[INFO] orthophoto_resolution: 1.0
[INFO] pc_classify: False
[INFO] pc_copc: False
[INFO] pc_csv: False
[INFO] pc_ept: True
[INFO] pc_filter: 0.0
[INFO] pc_las: False
[INFO] pc_quality: ultra
[INFO] pc_rectify: False
[INFO] pc_sample: 0
[INFO] pc_skip_geometric: False
[INFO] pc_tile: False
[INFO] primary_band: auto
[INFO] project_path: /var/www/data
[INFO] radiometric_calibration: none
[INFO] rerun: None
[INFO] rerun_all: False
[INFO] rerun_from: None
[INFO] rolling_shutter: False
[INFO] rolling_shutter_readout: 0
[INFO] sfm_algorithm: incremental
[INFO] skip_3dmodel: False
[INFO] skip_band_alignment: False
[INFO] skip_orthophoto: False
[INFO] skip_report: False
[INFO] sky_removal: False
[INFO] sm_cluster: None
[INFO] sm_no_align: False
[INFO] smrf_scalar: 1.25
[INFO] smrf_slope: 0.15
[INFO] smrf_threshold: 0.5
[INFO] smrf_window: 18.0
[INFO] split: 999999
[INFO] split_image_groups: None
[INFO] split_overlap: 150
[INFO] texturing_keep_unseen_faces: False
[INFO] texturing_single_material: False
[INFO] texturing_skip_global_seam_leveling: False
[INFO] texturing_skip_local_seam_leveling: False
[INFO] tiles: False
[INFO] use_3dmesh: False
[INFO] use_exif: False
[INFO] use_fixed_camera_params: False
[INFO] use_hybrid_bundle_adjustment: False
[INFO] video_limit: 500
[INFO] video_resolution: 4000
[INFO] ==============
[INFO] Running dataset stage
[INFO] Loading dataset from: /var/www/data/70aea463-22a8-4c94-85ee-81ae8876fd40/images
[INFO] Loading 347 images
[INFO] Wrote images database: /var/www/data/70aea463-22a8-4c94-85ee-81ae8876fd40/images.json
[INFO] Found 347 usable images
[INFO] Finished dataset stage
[INFO] Running split stage
[INFO] Normal dataset, will process all at once.
[INFO] Finished split stage
[INFO] Running merge stage
[INFO] Normal dataset, nothing to merge.
[INFO] Finished merge stage
[INFO] Running opensfm stage
[INFO] Wrote camera_models_overrides.json to OpenSfM directory
[INFO] Maximum photo dimensions: 5472px
[INFO] Photo dimensions for feature extraction: 2736px
[INFO] nvidia-smi not found in PATH, using CPU
[INFO] Altitude data detected, enabling it for GPS alignment
[INFO] [‘use_exif_size: no’, ‘flann_algorithm: KDTREE’, ‘feature_process_size: 2736’, ‘feature_min_frames: 64000’, ‘processes: 11’, ‘matching_gps_neighbors: 0’, ‘matching_gps_distance: 0’, ‘matching_graph_rounds: 50’, ‘optimize_camera_parameters: yes’, ‘reconstruction_algorithm: incremental’, ‘undistorted_image_format: tif’, ‘bundle_outlier_filtering_type: AUTO’, ‘sift_peak_threshold: 0.066’, ‘align_orientation_prior: vertical’, ‘triangulation_type: ROBUST’, ‘retriangulation_ratio: 2’, ‘matcher_type: FLANN’, ‘feature_type: SIFT’, ‘use_altitude_tag: yes’, ‘align_method: auto’, ‘local_bundle_radius: 0’]
[INFO] Wrote reference_lla.json
[INFO] running “/code/SuperBuild/install/bin/opensfm/bin/opensfm” detect_features “/var/www/data/70aea463-22a8-4c94-85ee-81ae8876fd40/opensfm”

[APOS80]

Hello

Can you share the dataset, I would like to analyse some things?

Besides to try to process I would like to look at image positions in QGis.

[alanterra]

Hi APOS80

The photos will be here (when Dropbox finishes synching, in probably 30 minutes): Dropbox - Photos - Simplify your life

There are 347 photos. This is a run where I did two passes, at 20° angles relative to each other, in order to calibrate my camera. I’ll probably remove the photos from my Dropbox in a few days.

Now that I think about it, I should probably just generate my point cloud from one of the passes, which might fix whatever is causing these holes in point cloud (since I will be using 1/2 the photos).

[Saijin_Naib]

Can you try without the pre-made lens calibration?

[alanterra]

Eventually, I’ll do that, Saijin. But I found in at least one case that using the pre-made lens calibration removed curvature in the point cloud, and if curvature is slipping into this project, it would invalidate my analysis.

[alanterra]

On the chance that Saijin is right, and the pre-calculated cameras.json is part of the problem, here is a link to that file, in case you want to play with it, APOS80. Dropbox - cameras.json - Simplify your life

[APOS80]

I’m trying my normal settings now, Hahog-Flann and fisheye. I don’t know much about the camera settings so I picked fisheye, it’s mostly safe to use.

I’ve noticed something in the images that might be a sign of movement, like the drone was moving when the images was captured. Did the drone move while capturing?

[alanterra]

Probably. I use Map Pilot Pro by Drones Made Easy to control the flight, and it tries to slow down the drone to reduce smear. I think that there is a setting to have the drone come to a halt for each photo, but I haven’t looked into it.

[APOS80]

Try Pix4D or GoodStation, they work well.

[Gordon]

It will take a few hours to download them all, but I have checked EXIF for one image and note 1/40 sec exposure at f/2.8, so I can see 2 reasons for the images being very soft. You are better off setting a higher ISO so you can stop the lens down a bit to produce sharper images, and use a faster shutter speed to reduce motion blur.

[APOS80]

1/40 is very low!

I managed to compute a cloud all images included, it was no holes. I’ll upload it when I get home again.

[Gordon]

Currently running flight 2 here, looks to be going ok so far, even with the soft images, some with shutter speed of only 1/20sec!

[Gordon]

Despite the soft images, the ortho and 3D textured model look ok with these settings:

[Parkridge Drive - 24/01/2023] 177images 04:24:13
|Options:|auto-boundary: true, dem-resolution: 4, dsm: true, gps-accuracy: 6, min-num-features: 18000, orthophoto-resolution: 2, pc-quality: high, use-3dmesh: true|
|Average GSD:|3.22 cm|
|Area:|32,164.57 m²|
|Reconstructed Points:|39,688,642|

[alanterra]

Thank you all for your help. I was also able to stitch these images using just one flight, and get a full point cloud.

I was pressed for time when I did this run, and I wanted to fly when it was overcast — so no shadows. But clearly I didn’t check the ISO and shutter speed carefully. The program I was using to control the drone kept putting up errors about shutter speed, but it wasn’t clear to me how to fix them.

So, I guess the smear in the photos contributed to the holes in the point cloud.

I have a lot to learn. Thanks again.

[APOS80]

You’re not alone, I’ve made many mistakes on the way. I’m not done learning even now.

The only way I know that works with bad light is an SLR, larger sensors has a lot less noise at higher iso.

[Gordon]

@alanterra

I use Map Pilot Pro

Does it have the option to pause for each photo? That would remove motion blur, but it will take longer to complete the flight though. Stopping down to f/4 or f/5.6 would sharpen the images somewhat too.

[smathermather]

@alanterra

But I found in at least one case that using the pre-made lens calibration removed curvature in the point cloud, and if curvature is slipping into this project

In my experience, you are more likely to introduce lens distortion by using a fixed lens for anything other than a true fixed-lens system.