Unexpected error: "Aborted (core dumped)" with large dataset

Hi everyone!

I’m running a process with 7,535 images taken at 120m of altitude with the mavic pro 2 (~3cm pixel size) on an amazon machine with 64vCPU (3.1 GHz) and 498GB of RAM and other 500GB of SSD SWAP (just in case…), with 1.5TB of hard drive available (SSD). I’m running on ubuntu server 18.04, using the following command:

sudo docker run -it --rm -v $(pwd)/images:/code/images -v $(pwd)/opensfm:/code/opensfm -v $(pwd)/odm_meshing:/code/odm_meshing -v $(pwd)/odm_texturing:/code/odm_texturing -v $(pwd)/odm_georeferencing:/code/odm_georeferencing -v $(pwd)/odm_orthophoto:/code/odm_orthophoto opendronemap/odm:0.9.1 --min-num-features 18000 --matcher-neighbors 24 --camera-lens brown --use-opensfm-dense --ignore-gsd --texturing-tone-mapping gamma --orthophoto-resolution 5 --verbose

I use odm v0.9.1 because it seems to work faster and with the same quality as newer versions (for our data at least). After ~3.5 days, at the reconstruction step (or immediately after: python /code/SuperBuild/src/opensfm/bin/opensfm reconstruct), I got this error:

image

It seems that I never run out of memory or disk space (and never used more than 12MB of SWAP). I have run several smaller process with a subset of ~2000 images each one, on smaller machines, without trouble (and in some cases using a lot of SWAP memory).

What could it be the reason? I re-launch the process and odm retake it at the reconstruction step to see what happens.

Thanks in advance!
Álvaro,

1 Like

Mm, I wonder if this works if you pass --use-hybrid-bundle-adjustment ?