Moving the ODM build system to pixi

A month has gone by since I shared the plan for the build system and I wanted to give you all an update.

Pixi

The move to pixi is largely completed with the native builds for Linux (x86, aarch64), Windows, and Mac all compiling and passing CI. This should make it much easier to contribute to the project for everyone who wants to. It will also make it much easier to maintain long term.

The remaining tasks in the transition are all related to testing to make sure I haven’t lost or broken anything in the process. As part of the move I extended our CI system slightly which proves it compiles and loads but it doesn’t yet test the output.

The pull request is now up, although there are a few outstanding tasks to handle before its really ready to be merged. If anyone wants to try it out, it should be as easy as these four steps:

  1. Install pixi on your platform following the Installation - Pixi guide
  2. Clone the repository (or checkout the pixi branch if you already have the repo cloned)
    git clone -b pixi https://github.com/OpenDroneMap/ODM.git
  3. Build it
    pixi run build
  4. And finally run it using any options you need
    pixi run odm -- <any options you normally use> <dataset_name>

I am still a newbie when it comes to photogrammetry and so I will have to lean on the you folk as experts in the field to help with the validation of the output. If you have any free time please give it a go with your own datasets and let us know how it compares. If you encounter any differences, regardless of how minor they appear please bring them up here!

Next steps

While I expect the transition to pixi to still take some time, depending on how the testing goes, I want to briefly look at the next steps as well.

Firstly, I want to extract the benefits of the change as soon as possible by doing a pass on our dependencies, bumping them up to newer versions where possible. This promises improvements in accuracy while also improving our processing speed.

Secondly, and somewhat concurrently, I would like to add more tests to the CI pipeline to help us capture differences in outputs. Any automation here will make it much easier move quickly in the future.

Finally, I would aim to have another release as soon as things are stable and we have validated the changes. The sooner we do this the sooner every user of ODM can benefit from the improvements we have made.

And that’s the build system update for now, thanks everyone!

5 Likes

Fantastic. Still loving the simpler interface for pixi than vanilla docker. That’s not just a dev community improvement, but a significant end-user improvement.

Did some hack work to make OpenDroneMap Automated Testing Suite (oats) work with pixi. Not proper work, mind, but gets us a run with all the test data with pixi, and with my update from a few minutes ago, retains most of the features.

Some thought will have to go into how / whether we test versions directly in oats moving forward, or if that’s downstreamed to a CI process.


FTR, all completed a run on Linux successfully. Now just sifting through the data and will report back when sifting is complete.


Post forthcoming, but on the testing side, in addition to the work by @huey, he and I have been working on some synthetic datasets to really enable that testing work to have reference outputs:

3 Likes

Ignorant question on pixi – once the build is complete, is there an update process? Made changes and thought I could just build again, but running into problems with redoing patches. Ideally I wouldn’t have to clean it every time, but this is a new world to me (and apologies is this is an easy find in the docs – it’s late on a Friday :D).