i am trying to work out if the radiometric calibration which uses the sun sensor really does what it should.
I came accross the following function:
def get_sun_sensor(self):
if self.sun_sensor is not None:
# TODO: Presence of XMP:SunSensorExposureTime
# and XMP:SunSensorSensitivity might
# require additional logic. If these two tags are present,
# then sun_sensor is not in physical units?
return self.sun_sensor / 65535.0 # normalize uint16 (is this correct?)
If like @pierotofy states and is also stated in the pix4d-documentation),
the value of SunSensor is in physical units when SunSensorExposureTime AND SunSensorSensitivity is not present why would a normalisation be neccessary or what factor should be applied at all.
I noticed in my DJI Multispectal Dataset, that the Irradiance value (which is equal to the SunSensor-Value) was around 3000. I dont find any information about what unit this is in. When looking at the micasense code which the multispectral code of odm is based on i think there is no normalisation based on bitdepth of the pixelvalues:
I noticed the problem because the resulting reflectance maps after stiching with odm have pixelvalueranges from -4 to 20… shouldnt they be between 0 and 1? Also i still clearly see differences in brightness due to changeing lighting conditions in the reflectance maps.
Well I certainly can do some tests with the Drone itself. If I have time I also maybe have access to another spectral lightsensor from apogee. Do you have any experience contacting the DJi support about such questions directly? if we knew what the units are the solution could be easy.
So I found an official image processing guide from DJI.
From that i found that changes in both calculations, radiance and irradiance, would be necessary for the DJI P4 Multispectral:
1. Radiance
I see that ind ODM the computation is also based on the micasense library where the radiance for some reason is multiplied with pi during reflectance calculation.
But in the DJI documentation there is no mention of it.
and pCam_NIR can be obtained from the SensorGainAdjustment Tag which seems to be the same as a1 in the RadiometricCalibration Tag
Also the Exposuretime seems to be multiplied with 1e-6 and NIR_gain seems to be taken directly from the SensorGain Tag instead of calculating it via ISO. This makes a difference since somehow these values differ.
(In the ExifTags from an DJIP4MS Image the SensorGain Tag is 1.0 while when calculating from ISO would result in a gain of 2.0.)
I guess there is some conditional logic needed since it seems that these steps are different for micasense for example.
2. Irradiance
For the Irrandiane measurement from the sensor, the value given by the Irradiance Tag sems to be used directly without any Bitdepth normalisation. In Eq.6 the denominator for the individual reflectances is the product of NIR_LS and pLS_NIR which seems to be written directly to the Irradiance Tag.
Which BTW i also cant find any signs of in any of the micasense files. So maybe this normalisation can be skipped in general and does not need to be specific for DJI.
If i have time i will try making a pullrequest for this with my suggested approach but maybe you are faster to implement this because i am not sure where the best spot is to insert the conditional checks for the Dronemodel. Directly inside the functions? It could be a check for Make and CameraModel Tag.
Example ExifTags exported with exiftool from A NIR Band Image of the DJIP4Multispectral:
ExifTool Version Number : 11.88
File Name : DJI_1155.TIF
Directory : .
File Size : 4.0 MB
File Modification Date/Time : 2022:05:20 10:46:20+02:00
File Access Date/Time : 2022:05:23 13:51:57+02:00
File Inode Change Date/Time : 2022:05:20 10:46:20+02:00
File Permissions : rwxrwxrwx
File Type : TIFF
File Type Extension : tif
MIME Type : image/tiff
Exif Byte Order : Little-endian (Intel, II)
Subfile Type : Full-resolution image
Image Width : 1600
Image Height : 1300
Bits Per Sample : 16
Compression : Uncompressed
Photometric Interpretation : BlackIsZero
Image Description : DCIM\100MEDIA\DJI_1155.TIF
Camera Model Name : FC6360
Strip Offsets : 8
Orientation : Horizontal (normal)
Samples Per Pixel : 1
Rows Per Strip : 1300
Strip Byte Counts : 4160000
Software : v01.17.2029
Modify Date : 2022:04:02 11:44:59
About : DJI Meta Data
Format : image/TIF
Absolute Altitude : +2100.18
Relative Altitude : +229.10
Gimbal Roll Degree : +0.00
Gimbal Yaw Degree : -104.90
Gimbal Pitch Degree : -85.00
Flight Roll Degree : +0.90
Flight Yaw Degree : -104.90
Flight Pitch Degree : -2.90
Flight X Speed : -1.30
Flight Y Speed : -4.60
Flight Z Speed : -0.80
Cam Reverse : 0
Gimbal Reverse : 0
Self Data : Undefined
Calibrated Focal Length : 1913.333374
Calibrated Optical Center X : 825.932129
Calibrated Optical Center Y : 662.799316
Rtk Flag : 16
Rtk Src Type : 0
Rtk Std Lon : 1.03884
Rtk Std Lat : 0.93276
Rtk Std Hgt : 2.42923
Relative Optical Center X : 0.00000
Relative Optical Center Y : 0.00000
Dewarp Flag : 0
Dewarp Data : 2020-02-18;1939.5799561,1930.4200439,28.2880249,-14.8120117,-0.4054570,0.3237830,0.0009036,0.0009036,-0.2959730
Dewarp H Matrix : 1.0000000,-0.0000000,0.0000000,-0.0000000,1.0000000,0.0000000,-0.0000000,-0.0000000,1.0000000
Vignetting Flag : 0
Vignetting Data : 0.000218235, 1.20722e-6, -2.8676e-9, 5.1742e-12, -4.16853e-15, 1.36962e-18
LS type : 1
Package idx : 27
Cfg cnt : 1
Raw Data : 2002.000 2823.000 2786.000 3152.000 3194.000 2473.000 1999.000 2464.000 2367.000 2825.000 1220.000 424.000 572.000 1160.000 1798.000 1669.000 1998.000 1926.000
Band Name : NIR
Band Freq : 840(+/-26)nm
Irradiance : 3289.142
Sensor Gain : 1.000
Sensor Gain Adjustment : 0.890243
Sensor Index : 5
Capture UUID : 253db13bc0d11ec924fa068f77b445c
Drone ID : a068f77b445cfba4011fa088c0ab0490
Version : 7.0
Has Settings : False
Has Crop : False
Already Applied : False
Rig Name : FC6360
Central Wavelength : 840
Wavelength FWHM : 26
Model Type : perspective
Principal Point : 2.484864,1.905564
Perspective Focal Length : 5.74
Perspective Focal Length Units : mm
Perspective Distortion : -0.405457, 0.323783, -0.295973, 0.000904, 0.000904
Rig Camera Index : 5
Black Current : 4096
Sun Sensor : 3289.142
Sun Sensor Exposure Time : 0.364
Vignetting Polynomial : 2.182350e-04, 1.207220e-06, -2.867600e-09, 5.174200e-12, -4.168530e-15, 1.369620e-18
Vignetting Center : 825.932129, 662.799316
Is Normalized : 0
Irradiance Exposure Time : 0.364000
Irradiance Gain : 16
Irradiance Yaw : 0.000000
Irradiance Pitch : 0.000000
Irradiance Roll : 0.000000
Radiometric Calibration : 0.890243, 0.000000, 0.000000
Exposure Time : 1/743
F Number : 2.2
Exposure Program : Shutter speed priority AE
ISO Speed : 200
Exif Version : 0230
Date/Time Original : 2022:04:02 11:44:59
Create Date : 2022:04:02 11:44:59
Shutter Speed Value : 1
Aperture Value : 2.2
Exposure Compensation : 0
Max Aperture Value : 2.2
Metering Mode : Center-weighted average
Focal Length : 5.7 mm
Make : DJI
Speed X : -1.30
Speed Y : -4.60
Speed Z : -0.80
Pitch : -2.90
Yaw : -104.90
Roll : +0.90
Camera Pitch : -85.00
Camera Yaw : -104.90
Camera Roll : +0.00
Exif Image Width : 1600
Exif Image Height : 1300
Custom Rendered : Normal
Exposure Mode : Auto
Focal Length In 35mm Format : 40 mm
Serial Number : a068f77b445cfba4011fa088c0ab0490
GPS Version ID : 2.3.0.0
GPS Latitude Ref : South
GPS Longitude Ref : West
GPS Altitude Ref : Above Sea Level
Black Level Repeat Dim : 1 1
Black Level : 4096
Aperture : 2.2
Image Size : 1600x1300
Megapixels : 2.1
Scale Factor To 35 mm Equivalent: 7.0
Shutter Speed : 1/743
GPS Altitude : 2100.1 m Above Sea Level
GPS Latitude : 3 deg 58' 26.00" S
GPS Longitude : 79 deg 4' 27.10" W
Circle Of Confusion : 0.004 mm
Field Of View : 48.5 deg
Focal Length : 5.7 mm (35 mm equivalent: 40.0 mm)
GPS Position : 3 deg 58' 26.00" S, 79 deg 4' 27.10" W
Hyperfocal Distance : 3.47 m
I now saw, that there is an open pullrequest regarding this issue already.
I did not go trough it all to see if every aspect of my above comment is represented.
but it seems that still math.pi is used for the radiance.
The code I added in my pull request (Multispectral improvement 211224_) does not use math.pi.
There are many problems in “photo.py” and “multispectral.py” in the original ODM.
For example …
(1) The dark level of the sensor is not read correctly from the EXIF tag, and is set to zero in the original ODM.
For DJI, it needs to read xmp(‘Camera:BlackCurrent’).
(2) Sensor gain is not read correctly from the EXIF tag.
In DJI, tags(‘EXIF ISOSpeedRatings’) contains rounding errors.
xmp(’@drone-dji:SensorGain’) should be used.
(3) There is a problem in the handling of the “time zone” in the timestamp.
Then, solar elevation of the sun is not calculated correctly for DJI drones.
DJI use “local time”, and does not record timezone information in EXIF.
(4) In the sun altitude correction, degree to radian conversions are missing.
Some of these issues have already been fixed in my pull request, but
Some are currently being corrected on my end.
There are many fixes and I am hoping that they will be incorporated into the original ODM, little by little.
Yes, i found that in the if statement for DJI it not uses math.pi indeed.
I also wondered if the Blackcurrent is utilized correctly but i did not dig into that yet.
For the sun position the only workaround would be to either:
define the timezone when running odm
calculate timezone based on GPS
if no timezone is given dont calculate the horizontal_irradiance at all and only use the normal irradiance.
Up to a certain tilt angle of the drone i think the error might be way less in compairsion to the massivley incorrect sunangle calculated with the wrong timezone. Maybe we give warnings for images or flights where the sensor is not horizontally aligned.
Now i also understand why i often get stripes in my reflectance maps parralell to the movement of the drone.
What i also found in the DJI Documentation and mentioned above is that the exposure time should be multiplied with 1e-6 before usage. So this also should be taken into account.
Additionally what i am thinking is if the conditional handling for each drone and cameramodel should be done inside every function or if for each vendor should get its own set of functions. Not familiar enough with python to handle that correctly but i imagine with more CameraMakes getting support writing a seperate if statement for each gets quite confusing in the long run. But i dont have enough experience in softwaredev to know what a good way would be. Maybe thats something for refactoring later when we get it working.
Over the past month, I have made corrections, in my multispectral branch, to ensure that Solar-Elevation correction and the Incident-Angle correction of sunlight to the DLS sensor work correctly.
The correction work in my branch is still in progress.
In the future, I plan to add algorithms for the difference between sunny and cloudy weather.
