This is awesome. I was just searching the forums to see if anyone had done any platform benchmarking.
It would be awesome if someone could get someone like Puget Systems to do benchmark testing on different CPUs like what they do for other mainstream products: https://www.pugetsystems.com/all_articles.php?filter[]=Metashape
Intel vs AMD, Intel Core iX vs AMD Ryzan and/or Threadrippers, Intel Xeon vs AMD Epic.
Whoa! Can you reach out? That would be an amazing addition.
I’ll try, but since I don’t have any contacts there, I can’t make any promises. I’ve spoken to them about buying a system from them for modeling in Pix4D before, but since they didn’t have any government contracts, I had to look else where.
Cool! Yeah, it’s probably a cold call for any of us. Let us know how it goes.
Zoo Dataset
Amount of RAM: 188 GB
CPU type: 2 x Intel(R) Xeon(R) CPU E5-2630 v2 @ 2.60GHz, 24 cores total
Operating System and Version: Ubuntu Linux 20.04.1
ODM Version: 1.4.2
Using Docker? Y
Using Split/Merge or Cluster? None
What configuration options did you use? use-3dmesh: true, mesh-size: 300000, mesh-octree-depth: 11, depthmap-resolution: 1000
Date of your test: 09/23/2020
Total time: 01:48:05
We’ll see how it goes. I’ll hit them up on FB later too.
Well, I tried:
I’ve created a pull requests for a couple of benchmarks on my system. Please have a look at it and merge it if you feel its useful.
Ive tried to overhaul the repository, do have a look at it and merge it if you like it
Excellent! I’ll take a look.
Great additions, thank you. I merged them. I would still like to get a parser going which can auto-generate those summaries by dataset but until then I agree that removing from README is the right thing to do. Maybe someday when we’re feeling fancy and not busy we can update to something like https://www.tanksandtemples.org/leaderboard/
I’ll try to get it done
Sorry, I’m not suggesting you need to do it @manand881 - I made a start but it’s a background task so not going very quickly. Of course if you want to have a go at it that would be great.
no worries, it’ll be a good learning experience for me too
I had some time this week and pushed my little nodejs parser across the finish line. There’s definitely more that can be done here, but this now auto-generates benchmark data organized by dataset, and another file organized by ODM version. You can browse the raw data as before, and all three are linked from the repo README.
From README.md:
View benchmark data by dataset:
View benchmark data by ODM version:
View raw benchmark data:
I have a new ODM processing machine, so I updated the benchmarks repo with additional data from January 2021. I also made some adjustments on the benchmark data parser, to rearrange and expand the parsed output.
Piero or Stephen - Can you help me update the benchmark data on the main ODM website? This is somewhat out of date, and the links to the benchmark data files no longer work. This is usually the first result for “ODM benchmarks” in google et al, so I’d like to get it up to snuff. I’m not clear on how that page is generated. Maybe this is just a matter of pulling in an updated README?
The ODM benchmark page on opendronemap.org pulls the README.md from GitHub - OpenDroneMap/odm-benchmarks: Benchmark data index for OpenDroneMap and WebODM. So they should be the same?
To fix the links, make sure they are absolute URLs (if they are relative, they will be invalid). This can be fixed by changing directly the README.md.
Ah, cool. OK I can sort that out. Thanks Piero.
I thought I’d add some benchmarks I just ran on my new Dell XPS, I wasn’t sure about doing a pull request so thought I’d share my result here instead:
Aukerman,0h 11m,Y,-,32 GB,3200 MT/s,Intel i7-11800H,2.3 Ghz (4.6 Ghz Turbo boost),8,NVMe SSD,Windows 11,Native,1.9.11,N,Default,N,2022-02-1,[Alastair Wilson],NVIDIA GeForce RTX 3050Ti
Aukerman,0h 16m,Y,-,32 GB,3200 MT/s,Intel i7-11800H,2.3 Ghz (4.6 Ghz Turbo boost),8,NVMe SSD,Windows 11,Native,1.9.11,N,High Resolution,N,2022-02-1,[Alastair Wilson],NVIDIA GeForce RTX 3050Ti
Just edited my post, as using build 51 means GPU is now working, so I’ll re-run the benchmarks.
Wow awesome, It more great if you will re-run the benchmark by using GPU that comparing CPU vs GPU.
Hopefully I will update your testing to ODM benchmark later (pull github)
