Orthophoto of 30 thousand images

Sharu · 13 June 2022 18:58

Hello,
i have big dataset of 30000 images wishing to generate orthopohto. i tried split and merge flags on some of the images but it is not generating report and 3D model for the same?
what should be strategy to generate the orthomosaic for larger dataset. even i read about cluster nodeodm. but can i use odm and see the results?

Saijin_Naib · 13 June 2022 19:09

Not generating the 3D model is expected for split/merge. That is a known limitation.

I’m uncertain about the report, however.

ClusterODM will have the same limitation of local split/merge in regards to not producing a 3D model.

You can try tuning some quality parameters, resizing images, or potentially using our planar reconstruction pipeline (under the “Field” preset in WebODM).

smathermather · 13 June 2022 21:22

Report generation is also not supported with split merge at this time.

Sharu · 14 June 2022 03:56

ok, what exactly happens in split and merge?
1.feature extraction
2.matching
3. reconstruction
4.split

are these the stpes followed?
if yes then how will it be for 30000 images? it may require more memory to execute all these stpes

smathermather · 14 June 2022 04:36

Hopefully, I can represent the process for multiple-node split-merge accurately enough. Where I’m wrong, I welcome corrections. You can glean much of this from:

github.com

OpenDroneMap/ODM/blob/master/stages/odm_app.py

import os, traceback, sys

from opendm import context
from opendm import types
from opendm import io
from opendm import system
from opendm import log

from stages.dataset import ODMLoadDatasetStage
from stages.run_opensfm import ODMOpenSfMStage
from stages.openmvs import ODMOpenMVSStage
from stages.odm_meshing import ODMeshingStage
from stages.mvstex import ODMMvsTexStage
from stages.odm_georeferencing import ODMGeoreferencingStage
from stages.odm_orthophoto import ODMOrthoPhotoStage
from stages.odm_dem import ODMDEMStage
from stages.odm_filterpoints import ODMFilterPoints
from stages.splitmerge import ODMSplitStage, ODMMergeStage

from stages.odm_report import ODMReport

This file has been truncated. show original

github.com

OpenDroneMap/ODM/blob/master/stages/splitmerge.py

import os
import shutil
import json
import yaml
from opendm import log
from opendm.osfm import OSFMContext, get_submodel_argv, get_submodel_paths, get_all_submodel_paths
from opendm import types
from opendm import io
from opendm import system
from opendm import orthophoto
from opendm.gcp import GCPFile
from opendm.dem import pdal, utils
from opendm.dem.merge import euclidean_merge_dems
from opensfm.large import metadataset
from opendm.cropper import Cropper
from opendm.concurrency import get_max_memory
from opendm.remote import LocalRemoteExecutor
from opendm.shots import merge_geojson_shots
from opendm import point_cloud
from opendm.utils import double_quote

This file has been truncated. show original

github.com

OpenDroneMap/ODM/blob/master/opendm/remote.py

import time
import datetime
import os
import sys
import threading
import signal
import zipfile
import glob
from opendm import log
from opendm import system
from opendm import config
from pyodm import Node, exceptions
from pyodm.utils import AtomicCounter
from pyodm.types import TaskStatus
from opendm.osfm import OSFMContext, get_submodel_args_dict, get_submodel_argv
from opendm.utils import double_quote

try:
    import queue
except ImportError:

This file has been truncated. show original

Processing starts on the lead node with some basic prep work: in addition to taking inventory of available data (photos, GCPs, geo.txt files, etc.) and settings, typically the lead node will proceed to submodel splitting with overlap between submodels, then the datasets get shipped off to the other nodes (and processed on the lead node as well) to do feature extraction, matching, and reconstruction (if local, then the feature extraction, matching, and reconstruction are all done locally instead, of course).

Those products are then returned to the lead node for alignment. OpenSfM traditionally could do this on the sparse point clouds or the camera pose, finding a sensible simple transform between each of the submodels from one or the other. I don’t know what the approach is now. At this stage much of the OpenSfM work is done, and the submodels are sent back out to the nodes for dense matching (OpenMVS), meshing, texturing, georeferencing, dem, and orthophoto.

Once those stages are complete on the nodes, then the data are tranfered back to the lead node one final time where they are combined: point clouds are merged (maybe clipped and merged, not sure), blended cutlines generated for the orthophotos allow for smooth mosaicking of the orthophoto, euclidian distance weighted blending of elevation models completes those, and meshes and final report are ignored.

Historically, the align step was the biggest memory problem, but I haven’t seen any issues with that recently. Most of the rest is super lightweight: not really any heavier in resource usage than processing a dataset roughly equal to the split size + the images in the overlap.

Sharu · 15 June 2022 13:07

thanks you so much! fantastically explained! can you please tell me how GCPs will work in this case,
I will split the dataset and run separately on different parallel nodes then will combine them together to form a single orthophoto (actually I m planning it to split 3000 images in one dataset so I will required 10 different nodes), in this case how do I split the GCPs file? or it will be done automatically?

Sharu · 16 June 2022 09:19

I have 5 local machines ready!
what if I spilt the images using split command and once I get all the submodel ready I will process the submodels individually and parallel on 5 different local machines. after all orthophotos ready Iwill merge them on local machine
can I do this?

Saijin_Naib · 16 June 2022 12:14

That is handled by ClusterODM and the split/merge pipeline. No need to do anything special aside from setting up ClusterODM and using split/merge.

system · 16 July 2022 12:15

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.