NodeODM log message - Capping memory usage

I noticed messages in NodeODM logs about memory usage capping and I’d like to understand why this capping happens and how the software determines what cap to set.
In those logs, just before the “Reading data for image…” messages there is always two messages,
“Capping memory usage to nn MB” and “Expecting to process nn images”.

Below are some examples.

From a computer with 64GB RAM, 128GB swap, 2TB SSD, 3GB GPU:
2022-03-01 05:36:57,534 INFO: Capping memory usage to ~ 24766.759765625 MB
2022-03-01 05:36:57,534 INFO: Expecting to process 200 images.
2022-03-01 04:54:34,976 INFO: Capping memory usage to ~ 25630.984375 MB
2022-03-01 04:54:34,976 INFO: Expecting to process 200 images.
2022-03-01 03:08:31,023 INFO: Capping memory usage to ~ 29401.572265625 MB
2022-03-01 03:08:31,023 INFO: Expecting to process 200 images.
and an example from a computer with 16GB RAM, 32GB swap, 1TB SSD, 3GB GPU:
2022-02-15 07:10:30,155 INFO: Capping memory usage to ~ 4397.548828125 MB
2022-02-15 07:10:30,155 INFO: Expecting to process 57 images.

The cap appears to be set much lower than the available CPU RAM, e.g. about half or a quarter of available RAM.
Why does that memory usage cap get set & how is the value calculated?


This is a great question!

I’ve searched the codebases of ODM, NodeODM, WebODM, OpenSFM, and OpenCV, and I can not find that string anywhere:

Capping memory usage to

I had hoped to elucidate things more, but my current understanding, taken from OpenSFM code, is this:
OpenSFM takes something like 90% of the free memory on Windows
Then, it sub-divides it into two halves, one for the parallel processing queue, and one for the processing task itself.

I will likely need a fact-check on this, however.

And I’d really love to know where that string is hiding :rofl:


It’s hiding in OpenMVS, I’m fairly sure.


Our friend was introduced into OpenSFM’s opensfm/actions/ on 2021-06-01:

It was removed from OpenSFM’s opensfm/actions/ on 2021-12-15:

and then added to OpenSFM’s opensfm/ on 2021-12-15:

whereupon it was removed from OpenSFM’s opensfm/ on 2022-02-21:

That being said, it looks like the calculation has not changed much in the time since the log message was added and then removed. My concern is whether or not I understood it correctly.

1 Like

Woah! That was a roller coaster.

1 Like

Thanks guys!
So at a high level it sounds like the memory cap is to prevent the feature extraction processing consuming all the available RAM on the computer?

1 Like

Seems so!

The commit message indicates that this is a means to try to manage memory deterministically since Python doesn’t let you do so.

Function is still intact, but the log message changed over time.

git log -S and git log -L are very neat and I just discovered them :rofl:

1 Like

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.