SFM Triangulation for DJI M300 Zenmuse P1

summitbri · 23 October 2022 14:50

Hello,

After adding the Zenmuse P1 sensor profile to the SFM database, SFM is still not picking up the YPR values from the XMP metadata to take advantage of the triangulation algorithm. Is there a separate list of DJI drones elsewhere in the code that would need to be updated (similar to the rolling shutter database)? I’ve been searching but haven’t found anything. Thoughts?

Here’s a sample of the full EXIF/XMP data for reference:

File File Type JPEG
File File Type Extension jpg
File MIME Type image/jpeg
File Exif Byte Order Little-endian (Intel, II)
File Image Width 8192
File Image Height 5460
File Encoding Process Baseline DCT, Huffman coding
File Bits Per Sample 8
File Color Components 3
File Y Cb Cr Sub Sampling YCbCr4:2:0 (2 2)
EXIF Image Description default
EXIF Make DJI
EXIF Camera Model Name ZenmuseP1
EXIF Orientation Horizontal (normal)
EXIF X Resolution 72
EXIF Y Resolution 72
EXIF Resolution Unit inches
EXIF Software 03.00.01.04
EXIF Modify Date 2022:10:16 14:09:20
EXIF Y Cb Cr Positioning Co-sited
EXIF Exposure Time 1/1000
EXIF F Number 5.0
EXIF Exposure Program Program AE
EXIF ISO 800
EXIF Sensitivity Type Recommended Exposure Index
EXIF Exif Version 0230
EXIF Date/Time Original 2022:10:16 14:09:20
EXIF Create Date 2022:10:16 14:09:20
EXIF Components Configuration Y, Cb, Cr, -
EXIF Shutter Speed Value 1/1000
EXIF Aperture Value 5.0
EXIF Exposure Compensation 0
EXIF Max Aperture Value 2.8
EXIF Metering Mode Center-weighted average
EXIF Light Source Fluorescent
EXIF Flash No Flash
EXIF Focal Length 35.0 mm
EXIF Flashpix Version 0100
EXIF Color Space sRGB
EXIF Exif Image Width 8192
EXIF Exif Image Height 5460
EXIF Interoperability Index R98 - DCF basic file (sRGB)
EXIF Interoperability Version 0100
EXIF File Source Digital Camera
EXIF Scene Type Directly photographed
EXIF Exposure Mode Auto
EXIF White Balance Auto
EXIF Digital Zoom Ratio 1
EXIF Focal Length In 35mm Format 35 mm
EXIF Scene Capture Type Standard
EXIF Gain Control None
EXIF Contrast High
EXIF Saturation Normal
EXIF Sharpness Hard
EXIF Device Setting Description (Binary data 4 bytes, use -b option to extract)
EXIF Serial Number 3XMDK3E001TYQX
EXIF Lens Info 35mm f/2.8
EXIF Lens Serial Number 02BY116G019P
EXIF GPS Version ID 2.3.0.0
EXIF GPS Latitude Ref North
EXIF GPS Latitude 38 deg 54’ 50.08"
EXIF GPS Longitude Ref West
EXIF GPS Longitude 106 deg 10’ 22.57"
EXIF GPS Altitude Ref Above Sea Level
EXIF GPS Altitude 2549.615 m
EXIF GPS Status Measurement Active
EXIF GPS Map Datum WGS-84
EXIF XP Comment 0.9.142
EXIF XP Keywords single
EXIF Compression JPEG
EXIF X Resolution 72
EXIF Y Resolution 72
EXIF Resolution Unit inches
EXIF Thumbnail Offset 25000
EXIF Thumbnail Length 11947
EXIF Thumbnail Image (Binary data 11947 bytes, use -b option to extract)
MakerNotes AE Debug Info (Binary data 4096 bytes, use -b option to extract)
MakerNotes AE Histogram Info (Binary data 4096 bytes, use -b option to extract)
MakerNotes AE Local Histogram (Binary data 2048 bytes, use -b option to extract)
MakerNotes AE Live View Histogram Info (Binary data 4096 bytes, use -b option to extract)
MakerNotes AE Live View Local Histogram (Binary data 2048 bytes, use -b option to extract)
MakerNotes AWB Debug Info (Binary data 4096 bytes, use -b option to extract)
MakerNotes AF Debug Info (Binary data 256 bytes, use -b option to extract)
MakerNotes Histogram disable,
MakerNotes Xidiri (Binary data 512 bytes, use -b option to extract)
MakerNotes Gimbal Degree -253,-450,0
MakerNotes Flight Degree -408,-91,69
MakerNotes ADJ Debug Info (Binary data 1024 bytes, use -b option to extract)
MakerNotes Sensor ID 3XMDK3E001TYQX
MakerNotes Flight Speed 73,69,0
MakerNotes Hyperlaps Debug Info (Binary data 8 bytes, use -b option to extract)
XMP XMP Toolkit Image::ExifTool 12.36
XMP About DJI Meta Data
XMP GPSXY Accuracy 0.03
XMP GPSZ Accuracy 0.03
XMP Format image/jpg
XMP Absolute Altitude +2549.61
XMP Altitude Type RtkAlt
XMP Cam Reverse 0
XMP Camera Serial Number 3XMDK3E001TYQX
XMP Drone Model Matrice 300 RTK
XMP Drone Serial Number 1ZNBJB400C003W
XMP Flight Pitch Degree -9.10
XMP Flight Roll Degree +6.90
XMP Flight X Speed 7.3
XMP Flight Y Speed 6.9
XMP Flight Yaw Degree -40.80
XMP Flight Z Speed 0.0
XMP Gimbal Pitch Degree -45.00
XMP Gimbal Reverse 0
XMP Gimbal Roll Degree +0.00
XMP Gimbal Yaw Degree -25.30
XMP GPS Latitude 38 deg 54’ 50.08" N
XMP GPS Longitude 106 deg 10’ 22.57" W
XMP Gps Status RTK
XMP Lens Serial Number 02BY116G019P
XMP Relative Altitude +28.965
XMP Rtk Diff Age 1.40000
XMP Rtk Flag 50
XMP Rtk Std Hgt 0.03
XMP Rtk Std Lat 0.03
XMP Rtk Std Lon 0.03
XMP Self Data
XMP Shutter Count 3675
XMP Shutter Type Mechanical
XMP Surveying Mode 1
XMP UTC At Exposure 2022:10:16 19:09:39.120001
XMP Version 1.4
XMP GPS Altitude 2549.615 m
XMP GPS Latitude 38 deg 54’ 50.08" N
XMP GPS Longitude 106 deg 10’ 22.57" W
XMP Make DJI
XMP Camera Model Name ZenmuseP1
XMP Create Date 2022:10:16 14:09:20
XMP Modify Date 2022:10:16 14:09:20
XMP Already Applied False
XMP Has Crop False
XMP Has Settings False
XMP Version 7.0
MPF MPF Version 0100
MPF Number Of Images 2
MPF MP Image Flags Representative image, Dependent parent image
MPF MP Image Format JPEG
MPF MP Image Type Baseline MP Primary Image
MPF MP Image Length 15913848
MPF MP Image Start 0
MPF Dependent Image 1 Entry Number 0
MPF Dependent Image 2 Entry Number 2
MPF MP Image Flags Dependent child image
MPF MP Image Format JPEG
MPF MP Image Type Large Thumbnail (VGA equivalent)
MPF MP Image Length 889076
MPF MP Image Start 15919854
MPF Dependent Image 1 Entry Number 0
MPF Dependent Image 2 Entry Number 0
MPF Image UID List (Binary data 66 bytes, use -b option to extract)
MPF Total Frames 1
MPF Preview Image (Binary data 889076 bytes, use -b option to extract)
Composite Aperture 5.0
Composite Image Size 8192x5460
Composite Megapixels 44.7
Composite Scale Factor To 35 mm Equivalent 1.0
Composite Shutter Speed 1/1000
Composite GPS Altitude 2549.6 m Above Sea Level
Composite GPS Latitude 38 deg 54’ 50.08" N
Composite GPS Longitude 106 deg 10’ 22.57" W
Composite GPS Latitude Ref North
Composite GPS Longitude Ref West
Composite Circle Of Confusion 0.030 mm
Composite Field Of View 54.4 deg
Composite Focal Length 35.0 mm (35 mm equivalent: 35.0 mm)
Composite GPS Position 38 deg 54’ 50.08" N, 106 deg 10’ 22.57" W
Composite Hyperfocal Distance 8.15 m
Composite Light Value 11.6

Saijin_Naib · 24 October 2022 02:44

I believe you’ll need to poke around in photo.py:

github.com

OpenDroneMap/ODM/blob/master/opendm/photo.py

import logging
import re
import os
import math

import exifread
import numpy as np
from six import string_types
from datetime import datetime, timedelta, timezone
import pytz

from opendm import io
from opendm import log
from opendm import system
from opendm.rollingshutter import get_rolling_shutter_readout
import xmltodict as x2d
from opendm import get_image_size
from xml.parsers.expat import ExpatError
from opensfm.sensors import sensor_data
from opensfm.geo import ecef_from_lla

This file has been truncated. show original

summitbri · 24 October 2022 14:25

Thank you sir!

As the P1 sensor pitch = -90 degrees at nadir, it looks like we might need to add zenmusep1 to this section of code as I’ve suggested below. Do you agree?

                # Normalize YPR conventions (assuming nadir camera)
                # Yaw: 0 --> top of image points north
                # Yaw: 90 --> top of image points east
                # Yaw: 270 --> top of image points west
                # Pitch: 0 --> nadir camera
                # Pitch: 90 --> camera is looking forward
                # Roll: 0 (assuming gimbal)
                if self.has_ypr():
                    if self.camera_make.lower() in ['dji', 'hasselblad']:
                        self.pitch = 90 + self.pitch

                     if self.camera_make.lower() in ['dji', 'zenmusep1']:
                        self.pitch = 90 + self.pitch  
             
                    if self.camera_make.lower() == 'sensefly':
                        self.roll *= -1

Saijin_Naib · 24 October 2022 20:32

Do you know if all the other axes match those listed in the codeblock you highlighted?

summitbri · 24 October 2022 20:43

Well that wasn’t the right fix, thanks @pierotofy for keeping me honest. Still scanning code to find a reason why the P1 YPR isn’t being picked up…

summitbri · 24 October 2022 22:24

Perhaps another hint here: P1 EXIF XMP data has both Flight & Gimball Yaw

XMP Flight Yaw Degree -40.80
XMP Flight Z Speed 0.0
XMP Gimbal Pitch Degree -45.00
XMP Gimbal Reverse 0
XMP Gimbal Roll Degree +0.00
XMP Gimbal Yaw Degree -25.30

In photo.py, this section of code seems to capture the Flight Yaw but not Gimbal Yaw. Thoughts?

                # OPK
                self.set_attr_from_xmp_tag('yaw', xtags, ['@drone-dji:FlightYawDegree', '@Camera:Yaw', 'Camera:Yaw'], float)
                self.set_attr_from_xmp_tag('pitch', xtags, ['@drone-dji:GimbalPitchDegree', '@Camera:Pitch', 'Camera:Pitch'], float)
                self.set_attr_from_xmp_tag('roll', xtags, ['@drone-dji:GimbalRollDegree', '@Camera:Roll', 'Camera:Roll'], float)

If that’s the issue, I’m probably not smart enough to fix it without breaking the FlightYawDegree for other drones.

Gordon · 24 October 2022 22:48

Do DJI use different conventions for different model drones?
For example, from photo.py

Pitch: 0 → nadir camera

Pitch: 90 → camera is looking forward

With the M2P Pitch 0 = looking forward, Pitch -90 is nadir

summitbri · 25 October 2022 15:03

The P1 sensor looks to be consistent with at least some of the other DJI drones:

Nadir pitch = -90
Gimbal Yaw facing north = 0

I haven’t looked closely at roll but assume it is consistent.

summitbri · 27 October 2022 22:24

So I’m all out of guesses related to photo.py. I’m starting to get the sense that the issue might be somewhere in the YPR–>OPK conversion in exif.py. Does anyone actually have this working?

Gordon · 28 October 2022 00:52

Maybe edit photo.py to correct that?

summitbri · 28 October 2022 14:39

I did suggest that in a PR, @pierotofy kept me honest and said that would end up applying the pitch correction twice.

system · 27 November 2022 14:40

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.