Camera position shift when processing high precision images with DOP

Hi All,

Recently I’m doing some tests with PPK data, which provides GPS along with horizontal accuracy(STD of xy) and vertical accuracy(STD of z). The STD values are usually less than 0.1m. The PPK information is provided with geo.txt.

However, I noticed sometimes the camera positions are all shifted towards one direction, the moving distance can be a lot larger than the processing without providing STD (using default DOP 10m instead). The measurement on the ground points is also less accurate compared to using default DOP. I used the opensfm’s viewer to inspect the results. As you can see in the 2 images below, white triangles are reconstructed camera positions, the other ends of the red lines are the GPS positions. The first image, which is reconstructed by providing source STD in geo.txt, has much longer red lines compared to the 2nd image, which is reconstructed by not providing source STD, hence using 10m DOP instead.

Camera positions using source STD
using_source_dop

Camera postions using default DOP
using_default_10m

I’m quite confused by the results. Doesn’t a smaller DOP value suppose to restrict the freedom of adjusting camera positions?

1 Like

Indeed, I agree with you and I would expect that using lower DOP would introduce no shift. Do you have GCPs ? Can you share some data for troubleshooting and reproducing the issue ?

1 Like

I don’t use GCPs for these datasets.
The dataset is from our customer, so I can only share it with you privately if you agree not to share it with others. Can I get your email or other sorts of contact?

1 Like

If you’re fully up to date, you might try using ORB as the --feature-type, since it is supposed to have smaller reprojection error/shift. However, it is currently experimental.

1 Like

I checked the GPS errors in stats.json, and the value is close to 0. In the _gps_errors function, it uses unbiased_gps for the comparison. I guess it’s not used in the viewer.
Which one should I trust? The GPS in the viewer looks more promising to me since I can align two reconstructions together, and the GPS positions are at the same place.

Screenshot from 2021-11-17 13-03-21

1 Like

I just saw 2.6.7 released yesterday. Will definitely try the new version.

1 Like

Indeed, in the stats.json, it takes into account the bias transform, which is non-zero when you have GCPs, which isn’t the case here. Can you share the report.pdf for both cases (DOP = PPK values and DOP = default). Thanks :pray:

1 Like

Yes, here are the reports
dop=ppk.pdf|attachment (3.6 MB)
dop=default.pdf|attachment (3.6 MB)

1 Like

Hi,

So looking at the report, I’d say that everything is fine and working as expected as GPS errors with PPK are much lower by 10x than with default. So I’d say then that there might an issue with the viewer itself ? Could you share the reconstruction.json only in both cases ?

Yann

2 Likes

OK, so I tried with some drone dataset (Red Rock https://dronemapper.com/sample_data/), with both default DOP (15 meters) and manually enforced DOP of 1 cm. I have consistent results between the report and the viewer :slight_smile:

DOP of 15 meters :
Screenshot 2021-11-19 at 12.29.07
Screenshot 2021-11-19 at 12.28.49

DOP of 1cm :
Screenshot 2021-11-19 at 12.29.19
Screenshot 2021-11-19 at 12.28.56

So, I’d say code-wise, everything is fine. So there might be probably something going with your dataset and/or processing settings maybe ?

Yann

2 Likes

Thanks for your patience.

I did more experiments, comparing odm257, odm267, and opensfm main branch.

5 experiments are executed several times to confirm the results

  1. reconstruct using ODM 257
  2. reconstruct using opensfm inside ODM 257 docker image
  3. reconstruct using ODM 267 (tag latest, currently tag 2.6.7 not working)
  4. reconstruct using opensfm inside ODM 267 docker image
  5. reconstruct using opensfm main branch
    The arguments I’m using for all the processing are
ODM:
--pc-csv --ignore-gsd --orthophoto-resolution 5 --dem-resolution 5 --use-3dmesh --matcher-distance 0 --matcher-neighbors 32 --min-num-features 16000 --feature-quality medium --pc-quality low

Opensfm(generated by ODM with above arguments):
align_method: auto
align_orientation_prior: vertical
bundle_outlier_filtering_type: AUTO
feature_min_frames: 16000
feature_process_size: 1368
feature_type: SIFT
flann_algorithm: KDTREE
local_bundle_radius: 0
matching_gps_distance: 0
matching_gps_neighbors: 32
optimize_camera_parameters: true
processes: 12
retriangulation_ratio: 2
triangulation_type: ROBUST
undistorted_image_format: tif
undistorted_image_max_size: 5472
use_altitude_tag: true
use_exif_size: false

Opensfm processing will use exif_override.json and config.yaml generated by ODM to guarantee the same config, let me know if I missed anything.

The issue of camera shifting only happens in experiment 1, and it happens consistently. It’s a bit strange as experiment 2 doesn’t have this issue.

But then I noticed another issue, the model elevations are different between ODM forked opensfm and the main branch. 2, 3, 4, are at the same elevation level, 5 is a few meters lower. And I’d say the reconstruction from 5(main branch) is much closer to the real value. Attached the reconstruction file for each experiment to check.

The same results happen to the (Red Rock https://dronemapper.com/sample_data/ ), I processed the 4th Ave Reservoir dataset with opensfm inside odm267 and opensfm main branch, gps-accuracy 0.01. Obtained different results as below.
Screenshot from 2021-11-19 14-11-27

@pierotofy, would you mind taking a look at this, seems it’s more related to ODM fork of opensfm

1 Like

Really interesting!

Thanks for digging deep and helping us reproduce and focus down where the behavior changes.

That is hugely helpful!

1 Like

Hi,

Thanks a lot for investigating.

Maybe looking at the values of the shots inside each reconstruction.json could tell us more about what’s happening (hypothesis is that OpenSfM processing didn’t change much and that metadata is the cause).

cat reconstruction.json | grep gps -n10

will output :

...
8561:                "gps_dop": 15.0,
8562:                "gps_position": [
8563-                    6.777438966330078,
8564-                    4.9679015424299,
8565-                    25.727375050075352
8566-                ],
...

With gps_position being the XYZ position of the GPS in the topocentric frame that is stored in the reference_lla.json

Yann

1 Like

Seems the elevation difference issue came from the projection type, by default opensfm uses “perspective” while ODM uses “brown” camera. This is written in code, so the config file doesn’t show this. I think it’s not an issue for the elevation difference between opensfm main branch and odm forked version.

I still couldn’t figure out the reason for camera shifting in 2.5.7. But I’ll start planning to upgrade to 2.6.7, and do more comparisons.

Thanks for all the suggestions!

1 Like

Hi,

Yes, indeed, brown camera has much more parameters, and depending on the geometry of the scene (flight capture angle, overlap, scene flatness), some parameters can have a lot of correlations with some others, in particular focal length/distortion/altitude and/or principal point/XY position : for example, focal length can compensate for altitude.

A way to overcome this is to avoid “unleashing” all the parameters at once in OpenSfM, and instead optimize for some parameters only at the very end.

Yann

2 Likes

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.