Cannot process dataset with AMD Windows 11 machine

cpo · April 27, 2024, 12:06am

Hi, I am processing a dataset of 400 (20 MP) images and it works well for everything I want to do on one windows 11 machine. I paid for the Windows installer, version 2.4.2 (ODM 3.5.0?), and I can get DTM, DSM, Orthophoto, FastOrthoPhoto to work just fine. The machine that it works on is a Windows 11 ThinkPad with Intel i5 11th gen, intel Xe onboard graphics with 32 GB of RAM. I tried it with several different options and all processed successfully. I did a fresh install with the windows installer, same version, on another Windows 11 machine which has AMD Ryzen 7 7840HS CPU, with onboard AMD Radeon 780M Graphics and 32 GB of RAM. Every time I try to process the same dataset on the AMD machine, I get “cannot process dataset”, even if I try to do the fast ortho photo option, which the other intel machine does in under 20 minutes. Does anyone know what the issue could be?

Thanks to anyone who may be able to help

Link to drone images:
https://app.box.com/s/g68tc2j1xk3742nqpg899rseddjm1ty0

Task output log is too long to paste in this window, so it can be found at the same link above.

uplanddog · April 27, 2024, 10:43am

Its hard to say why one works and the other doesnt, but things I would look at would be how much virtual memory you have allocated to the machine that it wont process on. Is there less free memory on the machine it wont process on?

400 images at 20 MP is probably eatting up quite a bit of ram and dipping into the virtual memory. If there isnt enough ODM could give that error.

cpo · April 27, 2024, 2:11pm

Hi, thanks for your reply. I installed with the windows installer, so I don’t think it’s running with virtual memory (it’s not running in docker or a virtual machine as far as I understand since I didn’t install docker). I tried again now and it failed after 4 seconds when I restarted the task. I was watching the RAM usage with task manager it less than 1/2 of the 32 GB of RAM was being used. The original log that I have uploaded at the link will give more information however I don’t know how to decipher it.

https://app.box.com/s/g68tc2j1xk3742nqpg899rseddjm1ty0

Let me know how to allocate more memory if I can. Thanks.

[INFO]    Fast orthophoto is turned on, automatically setting --skip-3dmodel
[INFO]    Initializing ODM 3.5.0 - Sat Apr 27 08:04:11  2024
[INFO]    ==============
[INFO]    3d_tiles: False
[INFO]    align: None
[INFO]    auto_boundary: True
[INFO]    auto_boundary_distance: 0
[INFO]    bg_removal: False
[INFO]    boundary: {}
[INFO]    build_overviews: False
[INFO]    camera_lens: auto
[INFO]    cameras: {}
[INFO]    cog: True
[INFO]    copy_to: None
[INFO]    crop: 3
[INFO]    dem_decimation: 1
[INFO]    dem_euclidean_map: False
[INFO]    dem_gapfill_steps: 3
[INFO]    dem_resolution: 5
[INFO]    dsm: False
[INFO]    dtm: False
[INFO]    end_with: odm_postprocess
[INFO]    fast_orthophoto: True
[INFO]    feature_quality: high
[INFO]    feature_type: dspsift
[INFO]    force_gps: False
[INFO]    gcp: None
[INFO]    geo: None
[INFO]    gltf: True
[INFO]    gps_accuracy: 3
[INFO]    ignore_gsd: False
[INFO]    matcher_neighbors: 0
[INFO]    matcher_order: 0
[INFO]    matcher_type: flann
[INFO]    max_concurrency: 16
[INFO]    merge: all
[INFO]    mesh_octree_depth: 11
[INFO]    mesh_size: 200000
[INFO]    min_num_features: 10000
[INFO]    name: bb1ee64b-b7c5-4276-b2b1-d5c311619aa1
[INFO]    no_gpu: False
[INFO]    optimize_disk_space: False
[INFO]    orthophoto_compression: DEFLATE
[INFO]    orthophoto_cutline: False
[INFO]    orthophoto_kmz: False
[INFO]    orthophoto_no_tiled: False
[INFO]    orthophoto_png: False
[INFO]    orthophoto_resolution: 5
[INFO]    pc_classify: False
[INFO]    pc_copc: False
[INFO]    pc_csv: False
[INFO]    pc_ept: True
[INFO]    pc_filter: 5
[INFO]    pc_las: False
[INFO]    pc_quality: medium
[INFO]    pc_rectify: False
[INFO]    pc_sample: 0
[INFO]    pc_skip_geometric: False
[INFO]    primary_band: auto
[INFO]    project_path: C:\WebODM\resources\app\apps\NodeODM\data\bb1ee64b-b7c5-4276-b2b1-d5c311619aa1
[INFO]    radiometric_calibration: none
[INFO]    rerun: None
[INFO]    rerun_all: False
[INFO]    rerun_from: ['odm_postprocess']
[INFO]    rolling_shutter: False
[INFO]    rolling_shutter_readout: 0
[INFO]    sfm_algorithm: incremental
[INFO]    sfm_no_partial: False
[INFO]    skip_3dmodel: True
[INFO]    skip_band_alignment: False
[INFO]    skip_orthophoto: False
[INFO]    skip_report: False
[INFO]    sky_removal: False
[INFO]    sm_cluster: None
[INFO]    sm_no_align: False
[INFO]    smrf_scalar: 1.25
[INFO]    smrf_slope: 0.15
[INFO]    smrf_threshold: 0.5
[INFO]    smrf_window: 18.0
[INFO]    split: 999999
[INFO]    split_image_groups: None
[INFO]    split_overlap: 150
[INFO]    texturing_keep_unseen_faces: False
[INFO]    texturing_single_material: False
[INFO]    texturing_skip_global_seam_leveling: False
[INFO]    tiles: False
[INFO]    use_3dmesh: False
[INFO]    use_exif: False
[INFO]    use_fixed_camera_params: False
[INFO]    use_hybrid_bundle_adjustment: False
[INFO]    video_limit: 500
[INFO]    video_resolution: 4000
[INFO]    ==============
[INFO]    Running dataset stage
[INFO]    Loading dataset from: C:\WebODM\resources\app\apps\NodeODM\data\bb1ee64b-b7c5-4276-b2b1-d5c311619aa1\images
[INFO]    Loading images database: C:\WebODM\resources\app\apps\NodeODM\data\bb1ee64b-b7c5-4276-b2b1-d5c311619aa1\images.json
[INFO]    Found 396 usable images
[INFO]    Coordinates file already exist: C:\WebODM\resources\app\apps\NodeODM\data\bb1ee64b-b7c5-4276-b2b1-d5c311619aa1\odm_georeferencing\coords.txt
[INFO]    Model geo file already exist: C:\WebODM\resources\app\apps\NodeODM\data\bb1ee64b-b7c5-4276-b2b1-d5c311619aa1\odm_georeferencing\odm_georeferencing_model_geo.txt
[INFO]    Finished dataset stage
[INFO]    Running split stage
[INFO]    Normal dataset, will process all at once.
[INFO]    Finished split stage
[INFO]    Running merge stage
[INFO]    Normal dataset, nothing to merge.
[INFO]    Finished merge stage
[INFO]    Running opensfm stage
[WARNING] C:\WebODM\resources\app\apps\NodeODM\data\bb1ee64b-b7c5-4276-b2b1-d5c311619aa1\opensfm\image_list.txt already exists, not rerunning OpenSfM setup
[WARNING] C:\WebODM\resources\app\apps\NodeODM\data\bb1ee64b-b7c5-4276-b2b1-d5c311619aa1\opensfm\exif already exists, not rerunning photo to metadata
[WARNING] Detect features already done: C:\WebODM\resources\app\apps\NodeODM\data\bb1ee64b-b7c5-4276-b2b1-d5c311619aa1\opensfm\features exists
[WARNING] Match features already done: C:\WebODM\resources\app\apps\NodeODM\data\bb1ee64b-b7c5-4276-b2b1-d5c311619aa1\opensfm\matches exists
[WARNING] Found a valid OpenSfM tracks file in: C:\WebODM\resources\app\apps\NodeODM\data\bb1ee64b-b7c5-4276-b2b1-d5c311619aa1\opensfm\tracks.csv
[WARNING] Found a valid OpenSfM reconstruction file in: C:\WebODM\resources\app\apps\NodeODM\data\bb1ee64b-b7c5-4276-b2b1-d5c311619aa1\opensfm\reconstruction.json
[INFO]    Already extracted cameras
[INFO]    Export reconstruction stats
[WARNING] Found existing reconstruction stats C:\WebODM\resources\app\apps\NodeODM\data\bb1ee64b-b7c5-4276-b2b1-d5c311619aa1\opensfm\stats\stats.json
[WARNING] Will skip exporting C:\WebODM\resources\app\apps\NodeODM\data\bb1ee64b-b7c5-4276-b2b1-d5c311619aa1\opensfm\reconstruction.geocoords.json
[INFO]    Undistorting C:\WebODM\resources\app\apps\NodeODM\data\bb1ee64b-b7c5-4276-b2b1-d5c311619aa1\opensfm ...
[WARNING] Already undistorted (nominal)
[WARNING] Found a valid OpenSfM NVM reconstruction file in: C:\WebODM\resources\app\apps\NodeODM\data\bb1ee64b-b7c5-4276-b2b1-d5c311619aa1\opensfm\undistorted/reconstruction.nvm
[WARNING] Found a valid PLY reconstruction in C:\WebODM\resources\app\apps\NodeODM\data\bb1ee64b-b7c5-4276-b2b1-d5c311619aa1\opensfm\reconstruction.ply
[INFO]    Finished opensfm stage
[INFO]    Running odm_filterpoints stage
[WARNING] Found a valid point cloud file in: C:\WebODM\resources\app\apps\NodeODM\data\bb1ee64b-b7c5-4276-b2b1-d5c311619aa1\odm_filterpoints\point_cloud.ply
[INFO]    Finished odm_filterpoints stage
[INFO]    Running odm_meshing stage
[INFO]    Writing ODM 2.5D Mesh file in: C:\WebODM\resources\app\apps\NodeODM\data\bb1ee64b-b7c5-4276-b2b1-d5c311619aa1\odm_meshing\odm_25dmesh.ply
[INFO]    ODM 2.5D DSM resolution: 0.9900000000000001
[INFO]    Created temporary directory: C:\WebODM\resources\app\apps\NodeODM\data\bb1ee64b-b7c5-4276-b2b1-d5c311619aa1\odm_meshing\tmp
[INFO]    Creating DSM for 2.5D mesh
[INFO]    running renderdem "C:\WebODM\resources\app\apps\NodeODM\data\bb1ee64b-b7c5-4276-b2b1-d5c311619aa1\odm_filterpoints\point_cloud.ply" --outdir "C:\WebODM\resources\app\apps\NodeODM\data\bb1ee64b-b7c5-4276-b2b1-d5c311619aa1\odm_meshing\tmp" --output-type max --radiuses 1.5550883635269477,2.1992270543883916,3.110176727053896 --resolution 7.920000000000001 --max-tiles 0 --decimation 1 --classification -1 --tile-size 4096 --force
Point cloud bounds are [minx: -390.829650879, maxx: 394.815856934, miny: -729.608032227, maxy: 726.243164062]
DEM resolution is (100, 184), max tile size is 4096, will split DEM generation into 1 tiles
r2.19923_x0_y0.tif
r3.11018_x0_y0.tif
r1.55509_x0_y0.tif
[INFO]    Generated 3 tiles
[INFO]    running gdalbuildvrt -input_file_list "C:\WebODM\resources\app\apps\NodeODM\data\bb1ee64b-b7c5-4276-b2b1-d5c311619aa1\odm_meshing\tmp\tiles_list.txt" "C:\WebODM\resources\app\apps\NodeODM\data\bb1ee64b-b7c5-4276-b2b1-d5c311619aa1\odm_meshing\tmp\tiles.vrt"
0...10...20...30...40...50...60...70...80...90...100 - done.
[INFO]    running gdal_translate -co NUM_THREADS=16 -co BIGTIFF=IF_SAFER -co COMPRESS=DEFLATE --config GDAL_CACHEMAX 28.4% -outsize 10% 0 "C:\WebODM\resources\app\apps\NodeODM\data\bb1ee64b-b7c5-4276-b2b1-d5c311619aa1\odm_meshing\tmp\tiles.vrt" "C:\WebODM\resources\app\apps\NodeODM\data\bb1ee64b-b7c5-4276-b2b1-d5c311619aa1\odm_meshing\tmp\tiles.small.tif"
0
...
10...20...30...40...50
..
.60...70...80
...90.
.
Input file size is 100, 184
0...10...20...30...40...50...60...70...80...90...100 - done.
[INFO]    running gdalbuildvrt -resolution highest -r bilinear "C:\WebODM\resources\app\apps\NodeODM\data\bb1ee64b-b7c5-4276-b2b1-d5c311619aa1\odm_meshing\tmp\merged.vrt" "C:\WebODM\resources\app\apps\NodeODM\data\bb1ee64b-b7c5-4276-b2b1-d5c311619aa1\odm_meshing\tmp\tiles.small_filled.tif" "C:\WebODM\resources\app\apps\NodeODM\data\bb1ee64b-b7c5-4276-b2b1-d5c311619aa1\odm_meshing\tmp\tiles.vrt"
0...10...20...30...40...50...60...70...80...90...100 - done.
[INFO]    running gdal_translate -co NUM_THREADS=16 -co TILED=YES -co BIGTIFF=IF_SAFER -co COMPRESS=DEFLATE --config GDAL_CACHEMAX 28.4% "C:\WebODM\resources\app\apps\NodeODM\data\bb1ee64b-b7c5-4276-b2b1-d5c311619aa1\odm_meshing\tmp\merged.vrt" "C:\WebODM\resources\app\apps\NodeODM\data\bb1ee64b-b7c5-4276-b2b1-d5c311619aa1\odm_meshing\tmp\tiles.tif"
Input file size is 100, 184
0...10...20...30...40...50...60...70...80...90...100 - done.
[INFO]    running fastrasterfilter "C:\WebODM\resources\app\apps\NodeODM\data\bb1ee64b-b7c5-4276-b2b1-d5c311619aa1\odm_meshing\tmp\tiles.tif" --output "C:\WebODM\resources\app\apps\NodeODM\data\bb1ee64b-b7c5-4276-b2b1-d5c311619aa1\odm_meshing\tmp\mesh_dsm.tif" --window-size 512 --radius 4 --co TILED=YES --co BIGTIFF=IF_SAFER --co COMPRESS=DEFLATE
Input: C:\WebODM\resources\app\apps\NodeODM\data\bb1ee64b-b7c5-4276-b2b1-d5c311619aa1\odm_meshing\tmp\tiles.tif
Size: 100x184
Blocks: 1
Smoothing...1... done
Wrote C:\WebODM\resources\app\apps\NodeODM\data\bb1ee64b-b7c5-4276-b2b1-d5c311619aa1\odm_meshing\tmp\mesh_dsm.tif
[INFO]    Completed smoothing to create C:\WebODM\resources\app\apps\NodeODM\data\bb1ee64b-b7c5-4276-b2b1-d5c311619aa1\odm_meshing\tmp\mesh_dsm.tif in 0:00:00.071741
[INFO]    Completed mesh_dsm.tif in 0:00:00.512585
[INFO]    Sampling points from DSM: C:\WebODM\resources\app\apps\NodeODM\data\bb1ee64b-b7c5-4276-b2b1-d5c311619aa1\odm_meshing\tmp\mesh_dsm.tif
[INFO]    running "C:\WebODM\resources\app\apps\ODM\SuperBuild\install\bin\dem2points" -inputFile "C:\WebODM\resources\app\apps\NodeODM\data\bb1ee64b-b7c5-4276-b2b1-d5c311619aa1\odm_meshing\tmp\mesh_dsm.tif" -outputFile "C:\WebODM\resources\app\apps\NodeODM\data\bb1ee64b-b7c5-4276-b2b1-d5c311619aa1\odm_meshing\tmp\dsm_points.ply" -skirtHeightThreshold 1.5 -skirtIncrements 0.2 -skirtHeightCap 100 -verbose
'"C:\WebODM\resources\app\apps\ODM\SuperBuild\install\bin\dem2points"' is not recognized as an internal or external command,
operable program or batch file.

===== Dumping Info for Geeks (developers need this to fix bugs) =====
Child returned 1
Traceback (most recent call last):
File "C:\WebODM\resources\app\apps\ODM\stages\odm_app.py", line 82, in execute
self.first_stage.run()
File "C:\WebODM\resources\app\apps\ODM\opendm\types.py", line 471, in run
self.next_stage.run(outputs)
File "C:\WebODM\resources\app\apps\ODM\opendm\types.py", line 471, in run
self.next_stage.run(outputs)
File "C:\WebODM\resources\app\apps\ODM\opendm\types.py", line 471, in run
self.next_stage.run(outputs)
[Previous line repeated 2 more times]
File "C:\WebODM\resources\app\apps\ODM\opendm\types.py", line 450, in run
self.process(self.args, outputs)
File "C:\WebODM\resources\app\apps\ODM\stages\odm_meshing.py", line 54, in process
mesh.create_25dmesh(tree.filtered_point_cloud, tree.odm_25dmesh,
File "C:\WebODM\resources\app\apps\ODM\opendm\mesh.py", line 41, in create_25dmesh
dsm_points = dem_to_points(os.path.join(tmp_directory, 'mesh_dsm.tif'), os.path.join(tmp_directory, 'dsm_points.ply'))
File "C:\WebODM\resources\app\apps\ODM\opendm\mesh.py", line 65, in dem_to_points
system.run('"{bin}" -inputFile "{infile}" '
File "C:\WebODM\resources\app\apps\ODM\opendm\system.py", line 112, in run
raise SubprocessException("Child returned {}".format(retcode), retcode)
opendm.system.SubprocessException: Child returned 1

===== Done, human-readable information to follow... =====

[ERROR]   Uh oh! Processing stopped because of strange values in the reconstruction. This is often a sign that the input data has some issues or the software cannot deal with it. Have you followed best practices for data acquisition? See https://docs.opendronemap.org/flying/
.100 - done.

uplanddog · April 27, 2024, 10:28pm

Even though the version installed is native windows does not mean it will not use virtual memory(pagefile in windows). however reading that it ran for such a short time then threw that error means something else is at play here. I ran your dataset on my system and it came out fine as well the dataset looks to be in fine shape.

By chance does this system have antivirus software other than windows defender? Any antivirus software other than windows defender is not supported.

cpo · April 28, 2024, 12:40pm

Both system I used, the one that worked and the one that didn’t, have bit defender anti-virus installed. The one that fails fails after 7 minutes. It only failed after a few seconds on that instance because I restarted a previously failed attempt. If I start it again from scratch, it takes around 7 minutes before it fails. Both system have an exclusion for the WebODM directory however the next time I try, I’ll disable the anti-virus completely. However now I’m not home where the machine is so I won’t likely be able to access it for another 2 or 3 weeks. Thanks for your help with this.

uplanddog · April 28, 2024, 1:02pm

Typically what has been discovered from other windows users is when the have a 3rd party anti-virus installed during installation of ODM parts of ODM do not get properly installed because those parts get blocked by the 3rd party software. Normally the anti-virus software and ODM need to uninstalled completely ,reboot the PC then reinstall ODM.

cpo · April 29, 2024, 1:53am

Thanks, i’ll try that when i get back home in a couple of weeks.

cpo · May 16, 2024, 4:51am

I did as you suggested and uninstalled WebODM, uninstalled Bitdefender anti-virus, and then reinstalled WebODM, and then ran with the same dataset and it processed it successfully on the first try. Thanks for your help!

uplanddog · May 16, 2024, 4:13pm

Glad it worked out

system · June 15, 2024, 4:14pm

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.