Output Comparison

I am planning to write a series of blog posts on comparing outputs for different software against WebODM. I am thinking of doing quantitative and qualitative analysis for Orthophoto, Point Cloud, DSM, DTM and 3D Model. Any suggestions on what methodology should I follow?
To start, I plan to get the datasets from https://cloud.pix4d.com/demo, run the same in WebODM and compare outputs.

1 Like

Well, there are advantages to both methods, right?

If your target audience for the comparison is an analyst, you’d want to have more quantitative analyses versus a consumer who would benefit from qualitative most.

Do you know if those datasets have RTK/PPK GCPs or other truth data? A quantitative analysis could look at XYZ distortion at those points, for instance.

Qualitative would be mostly visual analysis, you know? How does the ortho look (color, sharpness, blending, pixel choice [nadir VS oblique during reconstruction], etc. Same goes for the DSM and point clouds.

Yes, the datasets have GCPs. Would calculating RMS error at GCPs be a good comparison? What other quantities I can look for?

This is a screenshot from the quality report of a dataset.

For qualitative analysis, I am thinking about having an image slider comparison as on UAV arena. A side-by-side comparison can be shown for the ortho, DSM, DTM as well as screenshots of some views of point clouds and 3D mesh. Are there any other ways to show a comparison?

Btw what do you mean by pixel choice [nadir VS oblique during reconstruction]?

1 Like

You could possibly compare coloration values at given reference points to test the image blending and balancing algorithms, though without a calibration target it is hard to quantitatively compare…

Slider of the products seems best, IMO. Maybe also compare generation time, file export size/resolutions, etc?

Oh, if you look at a generated orthomosaic, you can sometimes see that certain parts of the image are reconstructed from pixels closer to nadir vs pixels from images that were more oblique. Hard to explain, but usually easiest to spot in vegetation.