Largest dataset so far… to come

Talked to a colleague about drone photogrammetry, where going to test it on a project.

I estimated it will be about 2500 images, largest dataset I will be taking care of.

I’ll probably have to use split for this, I don’t think my 64gig ram and 8 cores will manage it as a hole. And I also estimated it will take at least 24h to process.

Anny recommendations for a larger set like this?


Sounds exciting! Will you be able to share any screenshots or anything once it is done?

I think Gordon and Stephen will likely be your best resources for something like this at the moment.


Yes, I might publish something if everything works out.


You should be right at your limit. Set your swap == RAM or 1.5x RAM or even 2x RAM and you should be fine, as long as you aren’t turning anything up to ULTRA.

    "imageSizeMapping": [
        {"maxImages": 40, "slug": "s-2vcpu-2gb"},
        {"maxImages": 250, "slug": "s-4vcpu-8gb"},
        {"maxImages": 500, "slug": "s-6vcpu-16gb"},
        {"maxImages": 1500, "slug": "s-8vcpu-32gb"},
        {"maxImages": 2500, "slug": "s-16vcpu-64gb"},
        {"maxImages": 3500, "slug": "s-20vcpu-96gb"},
        {"maxImages": 5000, "slug": "s-24vcpu-128gb"}

69 hours into 1320 images on ultra quality (CPU feature extraction 12MB images too big for 4GB CPU memory), 96GB RAM and pagefile 145GB, I’ve seen 99% of RAM used, and 67% of RAM + Paging so far

1 Like

I need ultra, quality is key here.

Then you probably need more RAM. I haven’t done profiling on the differences, but almost all RAM failures we’ve seen on here have been folks turning those knobs. I recommend trying first medium, then high, then ultra, just so you have a sense at what quality output you are getting before it breaks.

1 Like

But doesn’t split reduce ram usage?

1 Like

For sure. You could split this into a few parts. Set your split size to maybe 1500 and turn the knobs up to ultra. But it’s good to run a the lower settings so you understand, when all is said and done, whether and when turning up the settings and increasing complexity through split-merge is worth it.


What would be a good split-overlap for split 1500?

1 Like

That really depends on your between-camera spacing. You probably want 3 or so cameras on each side of the overlap. So if your camera spacing is roughly 10m, you can set your overlap to 30-40 meters.


If only WebODM could calculate the right split based on the hardware. But that might be hard.


Agreed! This would be super useful, and probably not impossible. At that point, the flag might go away and it could automatically do it when needed.