Camera types, a small survey

smathermather · 12 December 2019 03:27

With recent version fo OpenDroneMap, it’s now possible to use different camera/camera lens types. Historically, we have supported perspective cameras. Now, thanks to improvements to OpenSfM, we have access to perspective, brown, fisheye, and spherical.

What does this mean for you? Well, with fisheye you can use your wide angle lenses, but with brown also known as Brown-Conrady, we really improve the photogrammetric output. @PCdurham mentioned this possibility over a year ago and was spot on.

We’re seeing incredible fidelity in our surface models and terrain models when using this. Here’s a view of the surface model of a zoo with Brown-Conrady:

And a surface model of a forest over a set of ledges:

.

So, try it out! It’s as easy as switching the camera-lens type to brown and processing.

I’d love to see your models. Should this be the default camera in OpenDroneMap? I think so, but want to see how it’s working for others.

smathermather · 12 December 2019 03:29

pierotofy · 12 December 2019 14:06

I’ve had mixed results, although I haven’t systematically documented them. Would also like to hear feedback on this.

smathermather · 12 December 2019 14:34

Yes, I have seen over-fitting, e.g. https://www.opendronemap.org/2019/12/self-calibration-of-cameras-from-drone-flights-part-3/

smathermather · 12 December 2019 15:57

Look at your Sand Key dataset:

smathermather · 12 December 2019 21:08

Comparison of the two over a zoo exhibit:
zoom0

smathermather · 12 December 2019 21:12

zoom1

pierotofy · 13 December 2019 20:02

A heads-up on the usage of the brown-conrady (which I forgot), we currently approximate the model and lose some precision during dense reconstruction because the NVM format does not have support for distinct focal_x and focal_y parameters (only focal). I wonder if we could improve results further by adding a camera model with more distortion coefficients but without the distinct focal lengths.

github.com

mapillary/OpenSfM/blob/master/opensfm/commands/export_visualsfm.py#L50




for shot in reconstruction.shots.values():
    q = tf.quaternion_from_matrix(shot.pose.get_rotation_matrix())
    o = shot.pose.get_origin()


    shot_size_cache[shot.id] = udata.undistorted_image_size(shot.id)
    shot_index[shot.id] = i
    i += 1


    if type(shot.camera) == types.BrownPerspectiveCamera:
        # Will aproximate Brown model, not optimal
        focal_normalized = shot.camera.focal_x
    else:
        focal_normalized = shot.camera.focal


    words = [
        self.image_path(shot.id, udata),
        focal_normalized * max(shot_size_cache[shot.id]),
        q[0], q[1], q[2], q[3],
        o[0], o[1], o[2],
        '0', '0',

(I added that comment in export_visualsfm)

pierotofy · 13 December 2019 20:36

MVE also has no support for two focal lengths. I wonder how the results compare with --use-opensfm-dense (which does support the model in full).

smathermather · 13 December 2019 23:11

Ahh, I forgot about this compromise, which was theoretical when we brought in MVE.

Interesting. I promised not to work this weekend, but maybe I can find some time on Monday to look.

pierotofy · 14 January 2020 02:46

Based on feedback (from other posts as well) so far I’d agree to make brown default, unless specified otherwise by the user. I could bring the change in with the multispectral branch.

smathermather · 14 January 2020 03:10

I never shared my results from --use-opensfm-dense: it is noisier than using brown and MVE. I’ll try to dig up those results and share… .

I think brown as default could be a good addition with the multispectral branch.

smathermather · 14 January 2020 03:11

Just for fun, check out this result using brown:

smathermather · 21 January 2020 04:46

klakar · 1 February 2020 13:56

Here’s a comparison where the only difference is “Brown” vs “Auto”.
Captured with DJI Phantom 3 4K

smathermather · 1 February 2020 16:57

That looks like a fantastic improvement.

pierotofy · 1 February 2020 22:24

This is with a modified brown camera where a single focal variable is used instead of two vs. perspective result (auto) on a difficult dataset.

anim

Tennis courts lines appear much more straight with the modified brown model.

Also visibly less noise in the DSM:

anim

smathermather · 2 February 2020 02:03

Oh my, this is spectacular.

Saijin_Naib · 4 February 2020 14:58

Amazing development!

I’d love to re-run my two test datasets (first two collects I ever performed, which usually bowl even in Pix4D, though WebODM Lightning did do better) against this latest engine update…

Is this live in WebODM Lightining, LiveODM, etc. yet?

I don’t have my own instance of ODM running yet.

Camera Model:
GoPro Hero4 Black
RGB and/or Blue/NIR NDVI
Peau Productions 3.97mm

zarg404 · 24 May 2023 15:32

What about auto parameter now ? With 3.1,7 documentation we can read

auto = Automatic selection of best camera projection model.

But if auto = brown, we should change the doc !

I am wondering what is the best camera model for a PH4 RTK without lens correction which has a 84° FOV

Here is an exemple of a landing pad in the corner of a picture
Capture d’écran 2023-05-24 à 17.25.13

I will compare fisheye and brown