Hi everyone,
I’m bringing over some thoughts from a recent GitHub issue (#2001) where I mentioned I’d share some broader feedback regarding image versioning and general usage. For context, I run ODM in a headless capacity as part of automated data pipelines on Kubernetes/GCP.
Here are my main pain points and some potential solutions:
1. Docker Image Tagging / Versioning
Currently, the :gpu tag on Docker Hub tracks master, which means it functions as a floating/edge tag. For those of us running automated production pipelines, pulling a floating tag can introduce sudden breaking changes. It would be incredibly helpful to publish stable, versioned tags for GPU images (e.g., opendronemap/odm:gpu-3.x.x) alongside the CPU versions to ensure infrastructure stability.
2. Developer Experience & Docker Build Times
I completely agree with @spwoodcock regarding the slow setup and building process. Currently, the Dockerfiles copy all local files right at the beginning of the build. This means that even a one-line change to the Python code invalidates the Docker cache and triggers a lengthy, full recompilation of the C++ SuperBuild.
I’ve been using a modified Dockerfile locally that splits this into three distinct cache layers:
- Run the C++ SuperBuild
- Install Python requirements (
requirements.txt)
- Copy the Python app code
This makes iterating and testing small Python changes significantly faster. If there’s interest from the maintainers in adopting this approach, I’d be very happy to submit a PR!
3. High Memory Consumption
Echoing what others have noted in this thread, the processing memory requirements can scale very aggressively. In our GKE cluster, we regularly have to route tasks to machines with 416 GiB of RAM for larger datasets. We currently manage this by dynamically provisioning larger machines based on the input dataset size, and using split-merge as another workaround, but any core optimisations to lower peak memory usage would be a massive win.
It would also be very helpful to have a better way to predict/extract RAM usage. That would make it much easier to set resource requests dynamically for processing tasks - we currently rely on a mapping based on dataset size which can be temperamental/inefficient.
Thanks to all the maintainers for building such a fantastic tool. I’m very happy to help out with PRs where I can!