Processing Node File Storage


#1

I spent yesterday separating WebODM from the processing node, as I found WebODM to be quite unresponsive while tasks were being processed (which I expected). But what I didn’t expect is that when the tasks are completed, a huge amount of data is left on the processing node. I expected that the results would have been returned to WebODM and not left on the processing node.

I had allocated 200GB to the processing node’s VM assuming that it was only temporary storage (enough space to render everything, then send it back to WebODM), but the 200GB was swallowed like it was nothing in just a few hours after running just a handful of tests through it.

My question is, why are files left on the processing node after they have been processed? Is there any need to keep them there, and is there a way to purge these files cleanly if not?


#2

They are kept there for 2 days by default, in case a person wants to restart a task from a certain step without having to reprocess everything.

You can change that using the flag:

--cleanup_tasks_after <number> Number of minutes that elapse before deleting finished and canceled tasks (default: 2880)