Is There a Way to Use zipurl with PyODM for Image Processing

General Help PyODM
1

Hi everyone,

I’m currently working with PyODM to process drone imagery using a NodeODM server. I noticed that NodeODM has an endpoint where you can pass a zipurl—a publicly accessible URL pointing to a ZIP file containing drone images—for processing. This is really useful for situations where images are hosted online, and I’d prefer not to upload them individually.

Nodeodm endpoint for using zipurl is mentioned here: NodeODM/docs/index.adoc at master · OpenDroneMap/NodeODM · GitHub

My question is: **Is there any way to use the zipurl parameter with PyODM

I’ve been looking through the documentation and haven’t found explicit support for this feature in PyODM, but since NodeODM can handle it, I’m hoping there’s a way to pass the zipurl from PyODM or a workaround I might have missed.

1 Like
2

It is not supported yet, but here is my simple local extension that allows just that. Maybe some proper programmer can implement this later to the pyodm.

import pyodm

#%% Connect to the NodeODM server

# Address of the NodeODM instance. Must start with "http://"
node_url = "http://localhost:3000"
# Create a node object
node = pyodm.Node.from_url(node_url)

# Make sure the node is usable for us.
try:
    # ask the status of the node
    node_info = node.info()
except Exception as err:
    raise ConnectionError(
        f"NodeODM server cannot be connected to in address: {node_url}"
    )

print(f"Connected to NodeODM server at address: {node_url}")
print("Node details:")
print(vars(node_info))

#%% Add custom create_task_using_zipurl() function to the node

def create_task_using_zipurl(self, zipurl, options={}, name=None, progress_callback=None, skip_post_processing=False, webhook=None, outputs=[], parallel_uploads=10, max_retries=5, retry_timeout=5):
    """Start processing a new task.
    At a minimum you need to pass a list of image paths. All other parameters are optional.
    
    >>> n = Node('localhost', 3000)
    >>> t = n.create_task_using_zipurl("https://my_web_service.com/images.zip", \
                      {'orthophoto-resolution': 2, 'dsm': True})
    >>> info = t.info()
    >>> info.status
    <TaskStatus.RUNNING: 20>
    >>> info.last_error
    ''
    >>> t.info().images_count
    2
    >>> t.output()[0:2]
    ['DJI_0131.JPG - DJI_0313.JPG has 1 candidate matches', 'DJI_0131.JPG - DJI_0177.JPG has 3 candidate matches']
    
    Args:
        zipurl (str): URL to a ZIP file containing the images.
        options (dict): options to use, for example {'orthophoto-resolution': 3, ...}
        name (str): name for the task
        progress_callback (function): callback reporting upload progress percentage
        skip_post_processing  (bool): When true, skips generation of map tiles, derivate assets, point cloud tiles.
        webhook (str): Optional URL to call when processing has ended (either successfully or unsuccessfully).
        outputs (list): Optional paths relative to the project directory that should be included in the all.zip result file, overriding the default behavior.
        parallel_uploads (int): Number of parallel uploads.
        max_retries (int): Number of attempts to make before giving up on a file upload.
        retry_timeout (int): Wait at least these many seconds before attempting to upload a file a second time, multiplied by the retry number.
    Returns:
        :func:`~Task`
    """
    # Do the pyodm imports
    from pyodm.types import NodeOption, NodeInfo, TaskInfo, TaskStatus
    from pyodm.exceptions import NodeConnectionError, NodeResponseError, NodeServerError, TaskFailedError, OdmError, RangeNotAvailableError
    from pyodm.utils import MultipartEncoder, options_to_json, AtomicCounter
    from requests_toolbelt.multipart import encoder

    if not self.version_greater_or_equal_than("1.4.0"):
        return self.create_task_fallback(files, options, name, progress_callback)
    
    fields = {
        'name': name,
        'options': options_to_json(options),
    }
    
    fields['zipurl'] = zipurl
    
    if skip_post_processing:
        fields['skipPostProcessing'] = 'true'
    
    if webhook is not None:
        fields['webhook'] = webhook
    
    if outputs:
        fields['outputs'] = json.dumps(outputs)
    
    e = MultipartEncoder(fields=fields)
    
    result = self.post('/task/new', data=e, headers={'Content-Type': e.content_type})
    if isinstance(result, dict) and 'error' in result:
        raise NodeResponseError(result['error'])
        
    return self.handle_task_new_response(result)    

# Bind the zipurl function to the pyodm node-object
from types import MethodType
node.create_task_using_zipurl = MethodType(create_task_using_zipurl, node)

#%% Send a task to the node using zipurl

# Define URL to a ZIP file containing the image files
# Optionally use allas.create_presigned_download_url() to create URL to non-public databucket.
zipurl = "https://myserver.com/myimages.zip"

print('Sending task with ZIPURL:')
# task = create_task_using_zipurl(node, zipurl)
task = node.create_task_using_zipurl(zipurl)
print(task.info())
1 Like