Let's compare ODM! [Data Challenge, earn a badge]

General Help
#1

Sometimes we get requests about “how does ODM compare to software XYZ”.

I have no answer to that.

I was thinking it could be cool to have 2-3 datasets processed in ODM as well as other software, place them onto a map and have a comparison tool to showcase differences. I think it would be interesting to see where we’re at in terms of quality and what still needs to be worked on.

This requires licenses to software or a trial (which some of you might have?). The three “most popular” ones that would be interesting to compare results to are:

  • DroneDeploy
  • Metashape
  • Pix4D

The three datasets that are up for comparison are:

Up for grab, limited edition badge to whoever posts one or more datasets processed in one or more of these software:

image

The output should contain:

  • Orthophoto
  • DSM
  • Point Cloud
  • Version of software used (or name of software + date of processing)

Processed using “medium quality” / default settings, 2 cm/px resolution.

Results

:mag: = Available
:white_check_mark: = Claimed
:hourglass: = In progress

DroneDeploy

aukerman: :white_check_mark: (@HAWK-i.us)
brighton_beach: :white_check_mark: (@HAWK-i.us)
sand_key: :white_check_mark: (@HAWK-i.us)

Metashape

aukerman: :white_check_mark: (@vantageds)
brighton_beach: :white_check_mark: (@vantageds)
sand_key: :mag:

Pix4D

aukerman: :white_check_mark: (@gpsman)
brighton_beach: :white_check_mark: (@gpsman)
sand_key: :white_check_mark: (@gpsman)

DroneMapper

aukerman: :white_check_mark: (@Saijin_Naib, @dronemapper-io)
brighton_beach: :white_check_mark: (@Saijin_Naib, @dronemapper-io)
sand_key: :white_check_mark: (@Saijin_Naib, @dronemapper-io)

9 Likes
#2
#3

Brand new novice to ODM here. Just starting to get acquainted, but I can run those 3 datasets through DroneDeploy. How do I report the outputs?

… Bob R.

4 Likes
#4

Simply place them on a Google Drive / Dropbox shared folder (or some other cloud storage service).

1 Like
#5

Will run them tomorrow and upload to Dropbox. Your Challenge asks the same question I’ve had rolling around for a long time. Can’t wait to see the results.

BTW – Am assuming that you are the author of The Missing Guide. I have to say that your work is stunning! Reaches the perfect balance for my needs. My only regret is that I started with “other” engines (DD and Pix4d) and had to claw my way through a super steep learning curve that could have been so much easier with ODM and your manual. Glad I can contribute a little to the Open cause.

4 Likes
#6

Glad you found the guide useful! :slight_smile: Appreciate the help.

1 Like
#7

We need some standardization of outputs for comparison. I’ve uploaded the photos and have the results ready to download and transfer to Dropbox as soon as we have the format and EPSG for the comparison.

The orthomosaic can be exported in three formats: GeoTIFF, JPG, and PDF, all in choice of three EPSG codes 3857 (Web Mercator), WGS 84, or EPSG 4326.

The elevation export has even more formats – same three above plus Raw DEM values, Contour DXF, and contour SHP.

Please advise which combinations you’d like to have for ODM comparison. … Bob R.

Having to edit a prior post because I’m a new user limited to 3 posts in a given thread. So this is intended to follow post #7 below:

"GeoTIFF, got it.

For the coordinate system, I’m confused. The output coordinate system can be any coordinate system desired and isn’t based on the location of the map. It’s usually chosen to match up to some existing drawing. At least that’s been my understanding. For example the Web Mercator is the Google Maps standard. Since we’re not trying to match up to anybody’s drawings, I propose that we use that coordinate system, i.e., Web Mercator EPSG 3857."

4/13 Edit (still limited to rookie post limit - need to post to other threads to get freed up).

The Drone Deploy results for the three sites are available here: https://www.dropbox.com/sh/40st2vl5ex8p1xw/AACWLAEiQ51lJ0tzIb1gqG60a?dl=0

Notes:

  • Exported as GeoTIFF format and WebMercator EPSG 3857 coord system for all
  • All are ZIP files. GeoTIFF unzips to 3 files: TIF, TFW, KML
  • 2 in/px resolution was available for all Orthomosaics, but not for Elevation exports
  • Elev exports that were not avail at 2 in/px were exported as Max Available and again as the next lower resolution.
  • Aukerman Elevation Max Available was 4.14 in/px. Have that one and a 8 in/px in Dropbox
  • SandKey Elevation Max Available was 3.14 in/px. Have that one and a 4 in/px in Dropbox
  • Brighton Elevation is at 4 in/px.

Pierotofy - If you want different export settings, please let me know. I’ll be finding some other threads to post to so I can get loose from the newbie limit. Hope this helps. Regards… R

2 Likes
#8

GeoTIFF, the nearest UTM projection would be best (https://mangomap.com/robertyoung/maps/69585/what-utm-zone-am-i-in-# and https://epsg.io/ can be used to look up the code), for the elevation model also GeoTIFF, same projection as the orthophoto. :+1:

1 Like
#9

I can run these through Esri’s Drone2Map if interested. I believe under the hood it uses Pix4D.

3 Likes
#10

Hi Pierotofy.
I could compare results with Pix4D. Unfortunately; I didn’t succeed to use ODM (run with docker) with a gcp_list.txt file.

2 Likes
#11

Yes there’s Pix4D behind Drone2Map (not sure which version though) so that would work!

1 Like
#12

That would be awesome. Make sure to announce which dataset(s) you’re processing, so that to avoid duplicate efforts.

1 Like
#13

That’s awesome, thank you!

p.s. was there an option to export the point cloud? I’m not sure if DD allows that.

1 Like
#14

I will process the 3 (aukerman, brighton_beach and sand_key). I have time with confinement!

2 Likes
#15

Well, I seem to have graduated to the big boys room – now I can post more than 3 to a thread. Aahhh…

Additional Note to Drone Deploy exports described above – the version was 2.59.0

Pierotofy - DroneDeploy provides different deliverables at different subscription levels, which is why I’m letting my Pro level expire. My work needs the too-expensive-level, but I can still export a 3D model in OBJ format. Don’t know if that would be of interest, but I’ll go ahead and do it for all three datasets and upload to Dropbox. Should be available at that same link by about noon Eastern Time USA. What is your time zone?

Regards… Bob R.

1 Like
#16

Sure! Thanks. Also Eastern time.

1 Like
#17

The 3D model zip files are in Dropbox for all 3 datasets along with the Orthomosaics and DEM’s from Drone Deploy version 2.59.0. All 3 datasets produced 3D models at 2 in/px.

2 Likes
#18

Happy to run any and all through Agisoft, which while not as popular is a quiet giant in accuracy.

4 Likes
#19

Brighton Beach - Metashape (1.5.2)

2cm GSD for Orthos
jpg compression used in Metashape
Default DEM for metashape (6.5cm GSD)
Default DSM for WebODM (default GSD)
No gradual selection used in Metashape, all default.
Highest accuracy alignment in Metashape - 40,000 key point, 10,000 tie point.
Medium pointcloud detail in Metashape
ESPG 32615 Projection (That’s what WebODM came out with as default, it was easier for me to change this is Metashape as i don’t know how to change WebODM to ESPG 4326)

https://drive.google.com/drive/folders/1id1USsr1y1N0RpffFtgkByMC3KDTSxjX?usp=sharing

3 Likes
#20

What version of Metashape is it? @vantageds

1 Like