Split and merge Errors

I am trying to get split and merge to work on my PC. I chose a dataset with 425 images from ODM data. and tried to run it on my PC. i knew my system would not be able to handle it in the default settings, which is why i chose that perticular dataset since i thought it was a good opportunity to learn how to use split and merge to try and process datasets larger than i normally can. i read the ODM cookbook and some other posts on how to use split and merge and got to work.
I chose the split size to be 40 and the overlap to be 30 thinking that its small enough for my system to be able to handle it.
as i started it process the dataset i ran into an error and opened the task output to see what was happening. I read through all the output line and found a perticular line that said 2021-07-07 14:54:29,289 INFO: Expecting to process 127 images.. this is odd. if it decided to process 127 images, then what happened to my input where i instructed it to just run 40 images at a time?
my system configuration is, ryzen 5 1500x, 24gb ram, windows 10 pro, docker.


please help me out here

1 Like

update, even though the process crashed a long time ago, ODM is still using a lot of system resources. If its not processing anything, then why does it need so much of my systems resources?

1 Like

What specific executable is still running? It might still be working in part of the pipeline.

ODM is still running in docker. That’s whats taking up my system resources. i don’t know whats happening in the pipeline once its thrown an error at me and shown me that the process has failed.

1 Like

Should be safe to kill docker and restart it then.

1 Like

I understand. Thats what i did but my main issue is with split and merge. can you please guide me on that ?

1 Like

What’s the largest dataset you can typically process?

1 Like

130 Images

1 Like

I thought I had replied. My apologies for the delay: when you specify the split size plus the overlap for the splits, that total area gives you what I’ll call the batch size. So a split of 40 plus 30 meters of overlap results in a batch of around ~140 which is larger than your machine can handle.

I would recommend reducing your batch size and potentially your overlap so that your batches are small enough to process.

3 Likes

how can i make these calculations myself? can you please teach me the math behind it?

1 Like

It’s probably easier to do trial and error. Turn your overlap down to 20 and your batch size down to 30 and it will probably process.

3 Likes

ill do it right now and let you know about what happens

3 Likes


it failed with the same error in the exact same way

1 Like

Oh, interesting. Can you scale back parameters any? feature-quality is a huge memory hog…

all of them are at default values. I don’t mind scaling back the feature quality but i don’t see how it bears a co relation with the number of images that it is choosing to process.

1 Like

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.