Yann Noutary, long time maintainer of OpenSfM (a core ODM upstream library) and periodic epic contributor to ODM proper, has started a new job that prevents him continuing to work on each. In his last months and days before his new job, he worked on a few different problems, from bug fixes to deeper computational efficiency, and wrote a nice email summarizing that work to hand off to Leo and the rest of us as he dives in to the new adventure.
He agreed that I could share that here, as it contains a bit of the history of OpenSfM, provides us the challenge of being maintainers of the project from now on, and lists the improvements he’s thought about, tested, or implemented in OpenSfM. For our convenience, he’s also wrapped the ready to use portion of these in a pull request which is now open here.
Hey everyone !
A small (actually bigger than expected) follow-up on the current OpenSfM status. A small recap’ on the official repository :
Created by Pau. Continued and supported while at Mapillary
Mapillary acquired by FB/Meta.
Github repo are synced with internal Meta monorepo. PR merge has to be triggered from inside Meta.
2025 : almost all the Mapillary team has left Meta.
There’s one person left inside (for how long ?). OpenSfM isn’t being actively developed anymore.
Only updates to OpenSfM will now be bots and linter.
So the main repo is “dead”.Still, there was a lots of things that could have been done to improve OpenSfM if we would have been actively working on it in these past years. So, I took advantage of my free time to improve a few things here and there. Everything is currently on my fork’s master branch : GitHub - YanNoun/OpenSfM. I also gave it a try to merge my repo into ODM/ODM branch. This is the ODM branch of my repo : GitHub - YanNoun/OpenSfM at ODM and here for a tentative PR :
Here’s a list :
Build system : we’ve moved to conda end of last year. Much easier to use, I definitely recommend. Plus, there’s much more packages. OpenSfM Docker images now also use Conda, so there’s a unique source-of-truth for dependencies setup.
- See here for the switch to Docker + Conda https://github.com/YanNoun/OpenSfM/commit/7d6bb480e00dc0e54257349948480aa2180bacd7
OPK angles : this is finally fixed here : https://github.com/YanNoun/OpenSfM/commit/dd18fe7bd53150aa736d5282dc8b37188c76f999
We will need to decide if this stays in OpenSfM, versus odm/photo.py fixups. In any case, we need to ensure this is consistent.
It still assume canonical orientation of camera being nadir. We might need add to options if people want to specify differents mounting options.
These angles have also been added to the quality report (and the rerun_export, see below) https://github.com/YanNoun/OpenSfM/commit/2045631e2b329e6afc54f6390d96e69061a44483
GCPs : the fix is similar to what was already here, it make uses of pyproj instead of using a rigid approximation https://github.com/YanNoun/OpenSfM/commit/446a4b06624f38f7dc9922cf737fd3e806e75636
I also added basic support for non-UTM (here WGS84 specify with its EPSG, so it’s very specific) : https://github.com/YanNoun/OpenSfM/commit/d1824b57c77d9365d9051350d6eb6d8987f8fc36
I also revamped the weighting scheme, hopefully it will avoids GCPs being too perfect (too biased) in ultra settings (lots of features) https://github.com/YanNoun/OpenSfM/commit/dd1578c31471a6cabb9b894e5e10f6ce57754fe0 and https://github.com/YanNoun/OpenSfM/commit/cd2452c7ad39b9730f2b9fb31c598038774de195
Speed improvements in incremental reconstruction
In general, it’s just much faster : a 7K image datasets now passes in less than 2 hours (entire SfM process). Reconstruction went from 24 hours to 50 minutes (Core 9 Ultra 285K, 24 processes)
It’s a combination of tracks decimation, careful code revamping, and better algorithms (see below)
METIS (ceres::NESDIS) https://github.com/YanNoun/OpenSfM/commit/82cf8c5801a5bfb48d6cb03526d9ad8ecfc78a39
That’s a classic one for optimizing bundle. It’s a library that does bundle’s solver factorization highly multithreaded and efficient.
It is now supported through conda, so Linux and MacOS should be fine.
HOWEVER, it is dependant if conda finds a ceres build in conda-forge built with METIS. In the medium term this will likely involves per-platform conda file and exact pinpointing of conda packages. I don’t have a Macbook anymore so I couldn’t try. For Windows native, I don’t know.
libtcmalloc https://github.com/YanNoun/OpenSfM/commit/7d6bb480e00dc0e54257349948480aa2180bacd7
Another classic one to improve memoru use with threads and also avoiding memoery fragmentation in long standing processes. It just divide the memory usage by 2 on large datasets !
It requires appending a LD_PRELOAD before running
It is suported on Linux for now. I don’t know for MacOS (couldn’t test). I know it should work on Windows native in theory (had it working 10 years ago at Autodesk and Pix), but the dependencies will have to be figured it out.
Rerun export : ./bin/opensfm export_rerun ~/data/DATASET/ --output my_reconstruction.rrd && rerun ./my_reconstruction.rrd
Just fun with Rerun of something I’ve always wanted to try : viewer + quality report alltogether
Not sue for long term but it is here. Struggle a bit with large datasets though.
And now the things I wish I had to work on :
Benchmarking : but I learnt yesterday that OATS was similar. Something where you could just launch with a list of datasets folder and a commit hash, it runs overnight all these datasets SfM, dense, compute quality report and outputs a big CSV so you can compare.
CUDA support :
For matching, I’ve tried RAPIDS cuVS, and on a RTX5090, it was only 20% faster than FLANN+24 Cores. It was brute-force though so better than FLANN. But without any x5 or x10, it’s not worth the hassle.
I wish I would have time to implement https://wiki.epfl.ch/edicpublic/documents/Candidacy%20exam/LDAHash.pdf I know it can work nicely with both GPU and CPU. Not super complicated, training can be done offline or online. We could even try using to project SIFT descriptors using the matrices provided here : https://documents.epfl.ch/groups/c/cv/cvlab-ldahash/www/ldahash1.0.tar.bz2 Then we have binary descriptor and it should match at blazing speeds.
For the bundle, CUDA support could improve, didn’t try though, but CUDA support in Ceres seems quite traightforward.
Large : splitting in large can be easily improved using pair_selection.get_representative_points that use GPS+OPK to get “points on the ground” that improve for oblique.
- We should think on using the EXIF “altitude_above_ground” in DJI, as it can improve the above.
Unfortunately, I won’t time to work on OpenSfM anymore as I’m starting a new job at ESRI.
Have fun, and thank you for keeping the open-source torch bright !
Yann