RAM requirements and cloud based processing

General Help
#1

Hello,

New to the software and docker. I have had some success on small projects (around 100 images) and ideally would like to attempt a larger project and compare the results from the major players (Pix4D, Drone Deploy) and WebODM. I’ve attempted to process a 450 image project of a construction site but while my computer easily processed the project in the Commercial packages I couldn’t get it to complete in WebODM.

The first attempt stopped early and it appeared to be a memory issue (16GB RAM) so I had it resize the images (not sure how that affects the process or results but I thought I’d try it). It ran almost to completion but stopped cold at the very end when creating the laz file. I suspect it was also a memory issue (based on my rudimentary skills with task manager) which brings me to my questions.

Is there a way to move processing into something like google cloud or another place where my machine is not the problem? Am I dreaming to think that my machine with 16GB of RAM will ever handle this using webODM? Do I have any alternatives other than giving up or buying new hardware in order to use this software?

I really hope there a fixes for this. I hate the idea of paying the big money to enjoy what is currently a hobby at this point.

Jamie

1 Like
#2

WebODM Lightning is by UAV4Geo and is a fairly priced paid/hosted cloud processing service like Pix4D’s Cloud. Bonus is all funds go towards improving ODM for all of us. I use it for my work processing as my work machine is 16GB (8GB free after all the endpoint security services), and sometimes it’s nice to just let it fly and collect results :blush:

Others here on the forum have worked with DigitalOcean and Amazon Web Services hosting of ODM, and I think even Azure. Give the search a go and see if they cover what you’re looking for.

As for RAM, more never hurts anything but your wallet.

If you want to learn more about what the parameters do, check the documentation of the tools, GitHub repos, documentation site, and optionally purchase the excellent WebODM: The Missing Guide book (Piero Toffanin/UAV4Geo), which also funds ODM development for all of us.

#3

I assume one of these is my best alternative to processing large projects? Is there some guidelines on how many images is too many when it comes to on board processing and RAM issues? Is it something that is currently being addressed by the contributors?

I really like the idea of cloud processing costs going back into the project and will definitely look into Lightning as my first choice.

Thanks for the info!

1 Like
#4

Yeah, depending upon how much work you want to do to set it all up yourself (or not).

We’ve all been trying to suss out how much RAM per megapixel/band/image, but it eludes me at present at least. Ongoing investigative work!

There is always work being done to improve accuracy, precision, speed, and efficiency, so yes, in general, this is something on the radar of the core contributors. There are experimental parameters to “tile” some of the early stages of processing which may help with RAM consumption.

1 Like
#5

Thanks I signed up and am currently uploading images to Lightning as I type. I’m probably going to pick up the book that explains the processing parameters as well. Love the control over the process you get here, unlike drone deploy where they decide what works, process, and then hold your projects hostage.

3 Likes
#6

If you get really squirrely, you can even contribute code, and the community/developers are the nicest I’ve come across and are incredibly helpful and supportive.

You seem to write well… Any desire to help with documentation or translation? Good writers are always needed :slight_smile:

#7

It will be awhile before I contribute any code. I wish I was better at it.

Documentation might be a potential contribution I could work on. I’m interested I guess not sure on how much time I have but interested.

2 Likes
#8

+1 for Open Drone Map The Missing Guide

It’s a really impressive book. Not only do you learn the software in great detail, but it’s the single best resource I’ve found for the “how” of the orthomosaic processing pipeline. I also really appreciate an avenue to support the project without using the cloud service - much of my imagery can’t be uploaded so it’s not an option.

2 Likes
#9

Thanks for your support! I’m sure you know about the “easy installers” for the various platforms so you can process locally with easier updates/installation, but just in case:
https://www.opendronemap.org/webodm/download/

GCP Editor Pro is also a great add-on:

#10

I’ve got those too! :smile:

I’m interested in the GCP Pro/WebODM integration only available with the paid version. We’ll see how it goes.

1 Like
#11

I haven’t messed with GCPs yet as I don’t have a decent GPS setup to collect them. I am interested in just gathering some 2 or 3D points from a map service if that’s possible. Does the GCP interface software allow you to collect GCPs from (a map service - Google Maps as an example) and use them in your project? I’d like my projects to at least match up OK with google or better if I get some real pts collected in the field

#12

I suppose you could, but just be aware that Google Maps is normally 3m+ (in good areas) off from real-world coordinates and you’re likely better using your phone to collect points using GPS Averaging than relying upon Google Maps.
Bing Maps tends to be far more accurate, but neither is anything close to “truth” data.

This is my current preferred Android app for GPS Averaging. With a long baseline (collect time) you can turn in some impressive results.

#13

Understood. Nothing beats really going out with a quality gps but at this point I would like my images to line up better when displayed against a google image or map backdrop and don’t need high accuracy.

Have you used GCPs with z values in your projects? I’ve attempted to take non-GCP projects Laz files into ArcScene and they are usually way below ground level and it took awhile for me to understand why the layer wasn’t displaying when it really was but below the default ground leve.

1 Like
#14

Yes, but not with ODM. In a past life doing commercial Agricultural survey work with commercia software, GCPs with XYZM were de rigueur.

Mmm… You can always tweak the point clouds using something like QGIS or lastools if you don’t have the right license level or add-ons for ArcGIS.

#15

Awesome. Z accuracy fails pretty miserably unless you have a pretty good GPS. I downloaded the GPS point app and see if does offer some elevation info but I’m sure it’s more x and y accurate the z accurate.

1 Like
#16

Yeah, Z accuracy on GPS is always a bit of a sore point, but averaging with a long baseline is really powerful. I hope it’ll meet your needs for the time being.