Hi everybody,
For a pretty large project (2040 x 20MP images GSD 2.0) I added an extra processing node (128GB RAM) into my WebODM installation, running the the project on the High Resolution preset. I was a bit surprised that this bigger machine also gave an error. (Mea Culpa: I didn’t read the log file), thought it had to do with allocated RAM and increased the RAM to 512GB. When this node also gave an error I decided to take a closer look at the errorlog.
I saw that the processing job was almost finished (after orthophoto and dsm) and found the following:
[INFO] running gdaldem hillshade "/var/www/data/28bfa28d-a479-4602-a8f8-325867e9ef50/odm_dem/dsm.preview.tif" "/var/www/data/28bfa28d-a479-4602-a8f8-325867e9ef50/odm_dem/dsm.previewhillshade.tif" -z 1.0 -s 1.0 -az 315.0 -alt 45.0
0...10...20...30...40...50...60...70...80...90...100 - done.
[INFO] running "/code/venv/bin/python3" "/code/opendm/tiles/hsv_merge.py" "/var/www/data/28bfa28d-a479-4602-a8f8-325867e9ef50/odm_dem/dsm.previewcolor.tif" "/var/www/data/28bfa28d-a479-4602-a8f8-325867e9ef50/odm_dem/dsm.previewhillshade.tif" "/var/www/data/28bfa28d-a479-4602-a8f8-325867e9ef50/odm_dem/dsm.previewcolored_hillshade.tif"
/code/venv/lib/python3.12/site-packages/osgeo/gdal.py:312: FutureWarning: Neither gdal.UseExceptions() nor gdal.DontUseExceptions() has been explicitly called. In GDAL 4.0, exceptions will be enabled by default.
warnings.warn(
Traceback (most recent call last):
File "/code/opendm/tiles/hsv_merge.py", line 198, in <module>
rScanline = rBand.ReadAsArray(0, i, hillband.XSize, 1, hillband.XSize, 1)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/code/venv/lib/python3.12/site-packages/osgeo/gdal.py", line 5283, in ReadAsArray
from osgeo import gdal_array
File "/code/venv/lib/python3.12/site-packages/osgeo/gdal_array.py", line 10, in <module>
from . import _gdal_array
ImportError: cannot import name '_gdal_array' from 'osgeo' (/code/venv/lib/python3.12/site-packages/osgeo/__init__.py)
[WARNING] Cannot generate colored hillshade: Child returned 1
[INFO] running gdal_translate -outsize 1400 0 -of png "None" "/var/www/data/28bfa28d-a479-4602-a8f8-325867e9ef50/opensfm/stats/dsm.png" --config GDAL_CACHEMAX 48.75%
ERROR 4: None: No such file or directory
===== Dumping Info for Geeks (developers need this to fix bugs) =====
Child returned 1
Traceback (most recent call last):
File "/code/stages/odm_app.py", line 82, in execute
self.first_stage.run()
File "/code/opendm/types.py", line 470, in run
self.next_stage.run(outputs)
File "/code/opendm/types.py", line 470, in run
self.next_stage.run(outputs)
File "/code/opendm/types.py", line 470, in run
self.next_stage.run(outputs)
[Previous line repeated 8 more times]
File "/code/opendm/types.py", line 449, in run
self.process(self.args, outputs)
File "/code/stages/odm_report.py", line 214, in process
system.run("gdal_translate -outsize {} 0 -of png \"{}\" \"{}\" --config GDAL_CACHEMAX {}%".format(image_target_size, colored_hillshade_dem, osfm_dem, get_max_memory()))
File "/code/opendm/system.py", line 112, in run
raise SubprocessException("Child returned {}".format(retcode), retcode)
opendm.system.SubprocessException: Child returned 1
===== Done, human-readable information to follow... =====
[ERROR] Uh oh! Processing stopped because of strange values in the reconstruction. This is often a sign that the input data has some issues or the software cannot deal with it. Have you followed best practices for data acquisition? See https://docs.opendronemap.org/flying/
100 - done.
With a little bit of help from ChatGPT (I know I know :-)) I got the suggestion that:
The failure happens in the report/preview generation stage, specifically when WebODM tries to create a colored hillshade preview of the DSM for the report:
So I decided to rerun the project with the flag: skip-report and voila… 2 minutes later the project was finished.
Small edit: This mapping was made with a DJI M3E
Here is Claude’s analysis: