GCPs not aligning, up to 0.5m error

No, I was using the GCP coordinates in .txt that were used to create the gcp.txt file used in the processing.

The alignment in the ortho does not change much when looking at ground_control_points.gpkg…
Grey = original GCP
Yellow = from gpkg file

image

image
image
image
image
image
image
image
image
image
image
image

1 Like

The GCP file might have some errors. For example, the tagged images did not seem to be properly centered for certain GCP points:

image

Triple check your ground control point file. Perhaps start with fewer points (5 GCPs are sufficient, with 3 images per GCP), so a total of 15 lines.

2 Likes

What kind of sorcery is this?

I was not able to replicate your screen, this is what I got using Lightning online GCP Interface:
image

However, this is what I get using WebODM for Windows GCPi, seens perfect, this is tool I used:

DJI_0692.JPG
image

DJI)0694.JPG
image

I also get the same result when using the client WebODM Lightning.

I will create a new gcp file using the Lightning online tool and reprocess. Could you try to replicate my results using both WebODM for Windows and the WebODM Lightning client, please?

Looks like a few images are missing from your upload to Google Drive. They comprise 2/3 images for tagging GCP01, so it should be dropped from the gcp_list.txt
image

I’m also finding, like Piero, that some of the tagged pixel locations can be pretty varied. Manually tagging is hard, though, so it isn’t super uncommon.
image

Yep… I haven’t developed POSM GCPi, so I can’t really be sure why it didn’t tag the images correctly. I would trust the GCP tool on Lightning better. (Disclaimer: my company, UAV4GEO, wrote the tool).

1 Like

I still don’t understand why I see such a difference when opening in Lightning a file originally created in POSM GCPi…

Anyway, I created a new gpc file, I see improvement, the rGCP error in the report looks good, however they are still not aligned!

70000.PNG
70001.PNG
70002.PNG
70003.PNG
70004.PNG
70005.PNG
70006.PNG
70007.PNG
70008.PNG
70009.PNG
700010.PNG
700011.PNG

Checking in QGIS:
Grey = original GCP
Yellow = from gpkg file
70000
70001
70002
70003
70004
70005
70006
70007
70008
70009
70010
70011

image

report.pdf (9.3 MB)

Download link for the results:
https://wln2.sfo3.cdn.digitaloceanspaces.com/34011b34-a539-4dd3-a90f-287ea13ed0bb/all.zip

OBS: I have updated the dataset, number of photos rised from 390 to 433.
https://drive.google.com/drive/folders/1Oh6mckrOdT_2RvJAaEo7fHcV9_LxnVlH?usp=sharing

1 Like

0.05 meters is pretty good!

You’re probably not going to get a perfect fit, perhaps rolling shutter correction is a factor here and we don’t have support for that in ODM (yet). It seems like a challenging dataset and Pix4D is doing a better job here.

1 Like

Okay, thank you very much for the support! It’s not so bad and we did what we could. I totally agree rolling shutter could be a major factor and I am unsure about the accuracy of how the gcp were collected.

0.05m is good, right? How should I interpret this? Yellow dots seens to be 0.05m from the grey dot’s, but in the ortho, except by a few (3 or 4) gcps that are indeed inside the white paint, the rest are out of the center by 0.2-0.5m…

1 Like

0.05m is the “average” error (root mean square to be correct). Some GCPs align well, others don’t, on “average” they are off by 0.05m.

Definition of good depends on the requirements of the project. :slight_smile:

1 Like

Piero, forgive my ignorance… could you help me understand this “error table” given by the ground_control_points.gpkg file, please?

First, what exactly is the yellow points in my pictures? They are imported as “ground_control_points” from the .gpkg file. Why they don’t match the original GCP coordinates?

What’s the meaning of “error_x”, “error_y” and “error_z” in this table?

image

Take for example GCP 70002 (fid 3 unnamed-2), for me the white mark in the orthophoto is displaced basically only in the x-direction, so I expected a high “error_x” and near zero “error_y”, however this is not compatible with the table.

Compare this image bellow with geotag information against the field “observations_list” to be sure we didn’t mixed up the labels:
image

This is happening for all gcp, the reported error does not match visually the map.

1 Like

X/Y depends on your relative coordinate system (where’s the north?). This is not going to align to the X/Y of your map (not at least in the way it’s implemented in OpenSfM/ODM). That’s why we usually talk about “circular” error instead of X/Y error (it’s orientation independent). Hope this makes sense. This could be improved in the future.

2 Likes

Ok, taking the distance:

error_horizontal =(x_error^2+y^2)^0.5

match my observations using QGIS measure tool

We can close the topic!

2 Likes

No yet…

I suspect the GCP RMS error calculation is wrong, it must be, for sure we are much higher than 5cm.

On the previous presented I have calculated what I call “error_horizontal”, which is:

error_horizontal = (error_x^2+error_y^2)^0.5

image

Than I manually calculate the RMS of those values and got

RMSE_horizontal = 0.35m

So, I guess the calculation in the report must be wrong. Without further calculations, the reported 0.05m RMSE makes no sense when errors for each GCP is higher than 0.05m.

1 Like

FYI, GCP accuracy control has landed in OpenSfM : feat: expose horizontal/vertical GCP accuracy · mapillary/OpenSfM@54d3605 · GitHub

Regarding GCP RMS computation, you can find it here OpenSfM/stats.py at main · mapillary/OpenSfM · GitHub.

@pierotofy : OpenSfM always re-triangulate GCPs on-the-fly, and the triangulated value is never exposed anywhere. How does the table of @gm2gabriel compute GCPs errors, i.e. how does it triangulate GCPs ? I would suspect discrepancies to come from there.

The triangulation process of OpenSfM is purely geometric (midpoint algo), and there’s no refinement, we might improve it in the future, which might improve GCP error.

2 Likes

:clap: so exciting! :partying_face:

In short, we re-rerun triangulation using multiview.triangulate_gcp after reconstruction. One discrepancy for sure is due to the fact that stats.json is computed before georeferencing, whereas we retriangulate / recalculate the GCP values after georeferencing. I might have to recheck some assumptions and get back on this. Maybe it’s a bug, but I’m pretty sure this only causes a rotation shift, thus skewing X/Y error values but not the overall error.

1 Like

I think the difference in your results @gm2gabriel is that RMS in the report is not horizontal RMS, it’s the overall RMS (X/Y and Z) and because the error in Z is pretty small, it brings down the value.

A good estimate of horizontal accuracy is CE90 which is also included in the ODM/OpenSfM report:

2 Likes

I’m not familiar with the code, so I might say complete garbage :

         for rec in reconstructions:
                triangulated = multiview.triangulate_gcp(gcp, rec.shots, 1.0, 0.1)
                if triangulated is None:
                    continue
                else:
                    break

            if triangulated is None:
                continue

            triangulated_topocentric = np.dot(A.T, triangulated) 
            
            coordinates_topocentric = np.array(gcp.coordinates.value)
            coordinates = np.dot(A, coordinates_topocentric) + b
            triangulated = triangulated + b

                 result.append({
                'id': gcp.id,
                'observations': [obs.shot_id for obs in gcp.observations],
                'triangulated': triangulated,
                'coordinates': coordinates,
                'error': np.abs(triangulated_topocentric - coordinates_topocentric)
            })

So given the SfMproj4 transformation as A, b, we compute the error as :
e = | (A^t . triangulated) - gcp.coordinates |
Normally, raw triangulated and gcp.coordinates should be in the same coordinate system, at least that’s how we compute the error in the stats.json and what we take for assumption in OpenSfM (and what is minimized in the bundle). Could the A^t rotation introduce error ?

1 Like

Self correction, I must be wrong because I get a value of 0.33 by recomputing the RMS using the values from the ground_control_points.gpkg table.

1 Like

Don’t we all :laughing:

Correct. In the ODM code that should work similarly (reorganized from above):

triangulated_topocentric = np.dot(A.T, triangulated) 
coordinates_topocentric = np.array(gcp.coordinates.value)
error = np.abs(triangulated_topocentric - coordinates_topocentric)

Which should be in the same coordinate system.

I’m going to run some more tests in the upcoming days, there might be something else going on, I will post my findings.

The one thing that might be at fault here, is that we “switch” the reconstruction.json file from local to geographic coordinates (minus a UTM offset) at some point before we call multiview.triangulate_gcp, so perhaps we’re doing a double-rotation or missing a rotation somewhere in here.

1 Like

Yeah, I was checking the calculations, not able to found a possible combination using x, y or z errors to get a compatible RMSE with the report.

Here is the full report concerning GCP:
image

1 Like