The topic is originally posted on opensfm (#915), but I didn’t get an answer for a few days. I think I may get some help here.
Basically, I want to project ground control point to image to get the pixel coordinates of the control point. I believe this can relieve people from picking the gcps on the image one by one. Although there are some great work(e.g. Find_GCP ) on automatically detecting gcp with some patterns, but not all the gcps have these patterns, so it may fail.
Below are my findings for these days:
First, we need to transform the gcp coordinates to WGS84, we can get lat, lon, alt. Then use the following code to project the ground control point to image.
pt3D= reference.to_topocentric(lat,` lon, alt)
pt2D= shot.project(pt3D)
pt2D_px = cam.normalized_to_pixel_coordinates(pt2D)
then check if the pt2D_px is inside the camera frustum.
I only have one problem here is that the altitude of `pt3D` is NOT right, resulting in the wrong `pt2D_px `. But I don't know how to get the right altitude since the reference altitude is set to 0 in function `invent_reference_from_gps_and_gcp` of `dataset.py`, and the height of ground control point ususally near 0, so we always get ~0 altitude of pt3D.
I checked the point altitude of reconstruted point cloud using Meshlab. The point altitude has a value of -90 ~ -70 m。If we use this altitude to project gcp to image, we can get the right coordinates.
So how can I get the right altitude when transform the gcp to topocentric?
How do you envision this working (within?) the context of OpenDroneMap? A plugin? An upgrade to POSM GCPi? Feature for GCP Editor Pro (which has proximity-based image pre-filtering)?
Ahh, unfortunately I don’t understand things well enough to have an informed comment. Hopefully someone stops by shortly who can share a bit more insight with us.
The reference can be treated as offsets of the coordinates (not exactly true, but you get the idea), the 0 ref altitude means it won’t offset the input elevation much if the points are closed to the reference lat lon.
The problem is why your gcps have altitudes near 0 but your model has altitude -90 ~ -70. You might want to check the gcp errors first. The gcp_errors are calculated by this, it also uses reference.to_topocentric
Thanks for your reply. I know this function, which calls the triangulate_gcp, where the gcps should include observations. The observations in gcp are also the pixel coordinates of gcp on the image. This exactly what I want to get here. In other words, I want to project the gcp to image to automatically get the gcp observations, instead of mannualy picking the points on the image by using GCPi or .GCP Editor Pro.
If I understand you correctly, you want to process a dataset without GCP and then use it to retrieve GCP tagging directly.
Unfortunately, this is not achievable. If you can directly project GCP to images at the right location using the model aligned by GPS data, this means the GPS data is accurate enough. Then there is no need to use GCP (RTK/PPK drones for example). GPS data is not accurate in general, especially altitude. This is the problem you are facing I suppose. While your GCP’s altitudes are all near 0, your images’ GPS altitudes might be far below 0, which makes the model’s altitude -90 ~ -70m. You can check your image GPS to confirm.
What you can try is picking GCP in one image, and using the ray intersection with the model to obtain a 3D point, then use this 3d point to find projections on other images. In this way, each GCP only needs one image to pick. This can reduce 2/3 of manual work I suppose, but in the trade-off, it will cost the time of one extra opensfm reconstruction, which is not that fast, most of the time, the manual picking will be faster.
But I still have one question here. I know the GPS data is not accurate enough, but if I set bundle_compensate_gps_bias = True in opensfm config, the latitude and longitude of the resulted point cloud is quite accurate, even reconstruted without GCP. So I don’t understand why the altitude is absolutely wrong.
Just to mention that such functionality would be welcome in my case. And I would be totally willing to sacrifice a very low resolution sparse reconstruction in order to mark a point in one image and have it automatically placed in all the others in an approximate position.
I could never figure out why, I think a faulty internal geoid model in the drone makes the GPSAltitude label wrong, not independently. That is, it is not a random error.
My solution to process without GCPs and to have the model approximately well in altitude was to create a bash script that modifies the GPSAltitude tags based on the value of the RelativeAltitude tag and the altitude at takeoff, in cambiar_altura.sh (in Spanish, it was made for a local presentation).
Thanks Gabriel for your script, though I need google translator to understand the REAME. Well, it’s not a big problem.
If I understant you correctly, you mean GPSAltitude = RelativeAltitude + take-off altitude , right?
I know there are values of latitude, longitude and altitude in EXIF, and the altitude sometimes is not right, and should be set to the value of GPSAltitude, am I right?
It’s common, GPS altitude is less accurate than lat/lon. The altitude can be greatly different if you fly on different days. It will be more consistent in the same flight though, so it will properly capture the elevation changes
Yes. What the script does is overwrite the value of the GPSAltitude tag based on the sum of a constant (parameter -m, takeoff altitude) plus the value of the RelativeAltitude tag for each image. Also, it rewrites the GPSAltitudeRef tag with the value 0 (“Above Sea Level”).
It is a problem that we find when the flight is made with some models of drone in particular, and we do not consider that it is due to an inaccuracy of the GNSS receiver since it is a non-random error, it always shows a value about 100 meters less than the real one.
In addition, any receiver that resolves the position by pseudo random code, for example any cell phone, resolves it with much better accuracy than that.
Luckily, those same drones record the RelativeAltitude tag with the relative to takeoff altitude.
I think opensfm calculates the altitude of the point cloud based on the altitude in image exif using triangulation algorithm, though the former should be lower than the latter. So these two altitudes should have some connection, or relationship. I dont’t think the altitude of the point cloud is the real altitude of the points on earth, or relative to Sea Level. and opensfm is not able to do it.
For example, the altitude of the camera in my dataset is about 135 meters above ground, and the point on the ground should have the altitude about 0, but this value is -80 ~ -90 meters in the resulted point cloud.
In one word, no matter the altitude of the camera is right or wrong, the relative altitude between the ground and the camera is fixed.
Corret me if I’m wrong.
The Z coordinate of the points in the point cloud is the “absolute” altitude with respect to sea level, which is the same altitude reference that the cameras’ GPSAltitude exif tag has.
Any other coordinate system (relative to the position of the camera, relative to the position of a point, or totally arbitrary) is internal to the photogrammetric processing and ultimately the point cloud is brought to the above sea level-referenced coordinate system.
Hi Gabriel, I checked the image exif with exiftool, here is the output:
ExifTool Version Number : 11.88
File Name : DJI_0001.JPG
Directory : .
File Size : 6.1 MB
File Modification Date/Time : 2019:04:08 06:40:08+08:00
File Access Date/Time : 0000:00:00 00:00:00
File Inode Change Date/Time : 2022:04:26 14:24:59+08:00
File Permissions : rw-rw-r--
File Type : JPEG
File Type Extension : jpg
MIME Type : image/jpeg
Exif Byte Order : Little-endian (Intel, II)
Image Description : DCIM\100MEDIA\DJI_0001.JPG
Camera Model Name : FC550
Orientation : Horizontal (normal)
X Resolution : 72
Y Resolution : 72
Resolution Unit : inches
Software : v01.29.5379
Modify Date : 2019:04:01 14:29:54
Y Cb Cr Positioning : Centered
Exposure Time : 1/7692
F Number : 1.7
Exposure Program : Program AE
ISO : 169
Exif Version : 0230
Date/Time Original : 2019:04:01 14:29:54
Create Date : 2019:04:01 14:29:54
Components Configuration : -, Cr, Cb, Y
Compressed Bits Per Pixel : 3.034780494
Shutter Speed Value : 1/7675
Aperture Value : 1.7
Exposure Compensation : 0
Max Aperture Value : 1.8
Subject Distance : 0 m
Metering Mode : Center-weighted average
Light Source : Daylight
Flash : No flash function
Focal Length : 15.0 mm
Warning : [minor] Possibly incorrect maker notes offsets (fix by 1415?)
Make : DJI
Speed X : +0.00
Speed Y : +0.00
Speed Z : +0.00
Pitch : -5.00
Yaw : +179.80
Roll : +3.70
Camera Pitch : -60.00
Camera Yaw : +0.00
Camera Roll : +0.00
Flashpix Version : 0010
Color Space : sRGB
Exif Image Width : 4608
Exif Image Height : 3456
Interoperability Index : R98 - DCF basic file (sRGB)
Interoperability Version : 0100
Exposure Index : undef
File Source : Digital Camera
Scene Type : Unknown (0)
Custom Rendered : Normal
Exposure Mode : Auto
White Balance : Manual
Digital Zoom Ratio : undef
Focal Length In 35mm Format : 30 mm
Scene Capture Type : Standard
Gain Control : None
Contrast : Normal
Saturation : Normal
Sharpness : Normal
Device Setting Description : (Binary data 4 bytes, use -b option to extract)
Subject Distance Range : Unknown
Lens Make : UnKnown
Lens Model : DJI MFT 15mm F1.7 ASPH
GPS Version ID : 0.0.0.0
GPS Latitude Ref : North
GPS Longitude Ref : West
GPS Altitude Ref : Above Sea Level
XP Comment : 0.9.199
XP Keywords : N-MF
Compression : JPEG (old-style)
Thumbnail Offset : 41984
Thumbnail Length : 8473
About : DJI Meta Data
Format : image/jpeg
Absolute Altitude : +135.75
Relative Altitude : +89.90
Gimbal Roll Degree : +0.00
Gimbal Yaw Degree : +0.00
Gimbal Pitch Degree : -60.00
Flight Roll Degree : +3.70
Flight Yaw Degree : +179.80
Flight Pitch Degree : -5.00
Flight X Speed : +0.00
Flight Y Speed : +0.00
Flight Z Speed : +0.00
Cam Reverse : 0
Gimbal Reverse : 0
Version : 7.0
Has Settings : False
Has Crop : False
Already Applied : False
MPF Version : 0010
Number Of Images : 2
MP Image Flags : Dependent child image
MP Image Format : JPEG
MP Image Type : Large Thumbnail (VGA equivalent)
MP Image Length : 286114
MP Image Start : 6099997
Dependent Image 1 Entry Number : 0
Dependent Image 2 Entry Number : 0
Image UID List : (Binary data 66 bytes, use -b option to extract)
Total Frames : 1
Image Width : 4608
Image Height : 3456
Encoding Process : Baseline DCT, Huffman coding
Bits Per Sample : 8
Color Components : 3
Y Cb Cr Sub Sampling : YCbCr4:2:2 (2 1)
Aperture : 1.7
Image Size : 4608x3456
Megapixels : 15.9
Scale Factor To 35 mm Equivalent: 2.0
Shutter Speed : 1/7692
Thumbnail Image : (Binary data 8473 bytes, use -b option to extract)
GPS Altitude : 135.7 m Above Sea Level
GPS Latitude : 40 deg 28' 37.36" N
GPS Longitude : 86 deg 52' 43.55" W
Preview Image : (Binary data 286114 bytes, use -b option to extract)
Circle Of Confusion : 0.015 mm
Field Of View : 61.9 deg
Focal Length : 15.0 mm (35 mm equivalent: 30.0 mm)
GPS Position : 40 deg 28' 37.36" N, 86 deg 52' 43.55" W
Hyperfocal Distance : 8.81 m
Light Value : 13.7
Warning : [minor] Possibly incorrect maker notes offsets (fix by 1415?)
Thumbnail Offset : 41984
It seems Relative Altitude is the flight height, right? So I must make a mistake before, I thought the GPS Altitude is the flight height. Now I should rewrite the GPS Altitude If I know the altitude of the take-off point . I believe this the problem.
And I need to point out that the altitude of reference coordinates in reference _lla.json is set to 0 manually(I don’t know why the author does this), I believe the it should be set to alt.
GPSAltitude seems OK to me. It is relative to sea level.
RelativeAltitude seems OK to me, too. It is relative to take-off.
So, the point cloud generated by ODM should be approximately at Z=+45. I don’t know why your point cloud is at Z=-90, as if the cameras were at sea level.