Importing Images for Processing in WebODM API

As per API documentaion to import images for prcoessing

images = [
    ('images', ('image1.jpg', open('image1.jpg', 'rb'), 'image/jpg')), 
    ('images', ('image2.jpg', open('image2.jpg', 'rb'), 'image/jpg')),
    # ...

In my case webODM and nodeODM running in an EC2 instance and my 12GB images are stored in different instance. When I call the Create Task API by forming image array as above, the RAM usage goes up to 12GB (as open(‘img’) creates BufferedArray). So to process 30GB of data I need 30GB of RAM to make api call and pass Images Array.

Is there any way of optimizing or other way of doing ?
Else is there any better method to process images from s3 bucket in webODM API?

1 Like

You can upload images in chunks by making sure to pass a partial parameter when creating a new task: WebODM/ at 2643aef14b6fb8b67e495b64c286c1db0e2b429a · OpenDroneMap/WebODM · GitHub

Then just do multiple POST uploads.

There’s no S3 support at the moment. We’d welcome a contribution!

Can you share some details as to what you are building?


That was great feature. Is there is any example or documentation. I couldn’t able to find it here


Hi pierotofy,
Thanks for your response and this great feature. I am experinmenting like processing lot of images

I have tried to implement as per your answer I cannot figure out how to make chunks of multiple images. I tried with with one image chunk like below

     def read_in_chunks(file_object, chunk_size=65536):
           while True:
                 data =
                 if not data:
                 yield data

           def createImageChunks():
                webodm_url = "http://url:8000"
                resAuth ='/api/token-auth/',
                                data={'username': 'usr',
                                'password': 'pwd'}).json()
                token = resAuth['token']
                res ='/api/projects/{}/tasks/'.format(int('10')), 
                        headers={'Authorization': 'JWT {}'.format(token)},
                        data = {'partial':True}) .json()
                task_id = res['id']

                """ f = [open('./DJI_0003.JPG', 'rb'), open('./DJI_0004.JPG', 'rb'),  open('./DJI_0005.JPG', 'rb')] """
                 f = open('./DJI_0003.JPG', 'rb')
                index = 0

                offset = 0

                headers = {}

                for chunk in read_in_chunks(f):

                    offset = index + len(chunk)

                    index = offset


                        res ='/api/projects/{}/tasks/{}/upload/'.format(int('10'), task_id), 

                                headers={'Authorization': 'JWT {}'.format(token),

                                'Content-Type' : 'multipart/form-data', 


                                files = chunk) .json()  


                    except Exception as e:

                        print (e)


The res variable prints as No files uploaded.
what are the parameter to be use when calling upload . Is there any sample code or documentation on how to do chunked uploads and how to call upload api



This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.