New Project - DroneDB (request for thoughts)

Lately I’ve been thinking a lot about what’s inefficient about the current state-of-the-art for publishing/sharing aerial images. I think Google Drive / Dropbox / USB drives and zip files have served us well, but I think there’s room for improvement.

So I started working on https://github.com/uav4geo/DroneDB

I think raw images captured from a drone are currently underutilized in analysis and they are often lost in archives never to be looked up again after they have been processed.

When working with drone data, I (personally) want to be able to:

  • Quickly access my data or a subset of it (without downloading all of it)
  • View where my data was captured, at what time (accurately), what flight pattern was used, what was the image footprint and did I have enough overlap, ground sampling distance metrics
  • Recreate and display the flight
  • Compare different datasets
  • Check for errors, corrupt images
  • Generate reports about my dataset (to publish along with the dataset itself)
  • Share it (locally or via internet)
  • Add notes and labels to my images / folders.
  • Have the option to not use a cloud service, because trust and speed.

Just wondering what people’s thoughts are and if these points are shared by other folks in the drone industry (or if you have your own personal take)?

2 Likes

Piero,

I think that’s a fantastic idea. It’s actually something I had tried to setup In a manner for myself using an existing package but just couldn’t get it to run properly.

I’m assuming that you’re thinking along the lines of a web-ui again? Sort of Lightroom for drone and photogrammetry users.

The report generation would be brilliant. I actually find it quite awkward to go back and find out the parameters I used to capture the images.

In my case it would the sort of thing I’d be happy to run on my new NAS server.

Longer term would you consider including flight logs? I appreciate that parsing many different manufacturer formats could be a pain. It does however link in with the capture pattern, and could be expanded to be a self hosted version of of the likes of DroneLogBook or AirData.

1 Like

Hey @dsandson :hand:

Yes there will be a web component to access the database (along with a desktop application for managing the database also).

When you say “find out the parameters you used to capture”, which specifically do you mean (overlap? flight speed?)

Definitely looking to add support for flight logs (GCPs, orthophotos, elevation models and point clouds are in the plans as well). By chance would you have a copy of a few logs that you could share (I don’t have much experience with flight log formats)? Would be helpful to have a few examples.

Piero,

For flight parameters I’d say at a minimum flight speed, front lap, side lap, and any common image capture parameters (iso, shutter speed, white balance), perhaps with user fillable options to add in basic weather params (cloud cover, wind speed). And perhaps total flight time and batteries used if flight logs are added in.

I’ll dig out some flight logs. I have access to a DJI Phantom 3 Pro and a Yuneec H520 & E90 camera. One issue with logs you will come across is each mapping app for DJI products produces a slightly different log format. And theres quite a few different mapping apps!

@pierotofy

Link is below to as many different log types as I can can find. This includes some of the major DJI flight apps;

  • DJI Go (not for P4P or Mavic 2)
  • DJI GS Pro
  • Litchi
  • AutoPilot
  • Pix4D Capture
  • Map Pilot (MapsMadeEasy)

Also, Ulog & Tlog formats from the Yuneec H520

https://drive.google.com/open?id=1jWozcsacb_52NW3evAoLhHHND8Ov1eTt

1 Like

Fantastic @dsandson. @pierotofy — reach out via email, and we can send logs from sensefly, and probably some pixhawk logs if you need them and maybe mavic.

On what scale or how much of the colourspectrum from a raw does the WebODM use (if it could read raw images)?
Hence JPG/TIF vs Raw colour spectrum.
I know raw from working in lightroom you get much large room to play with wider range of colours, so i just assume a raw file would give more data to play with. But also huge files…

Sorry, i noticed this was not the topic. More kind of metadata

This is gold @dsandson :+1:

DJI flight logs are encrypted, but I like a good challenge… :man_scientist:

1 Like

Ah I downloaded them again from DroneLogBook. Perhaps that’s the issue. I’ll look for originals tomorrow.

Actually I think they are simply encrypted by the DJI apps. But if you find the originals double-check!

Edit: the format is detailed in Documenting the Format of DJI Log Files

Correction, they are “scrambled”, not encrypted.

Piero, thought I’d add that the Yuneec H520 runs the PX4 flight control stack, which is apparently open source.

Found the following python script on the PX4 GitHub page, might be of use;
https://github.com/PX4/pyulog

1 Like

This sounds like a great project! I have actually been looking for something similar.

Would you have any use of logs from P4RTK, P4P or Sensefly?

Hey @hugo :hand: actually yes! It would help implementing https://github.com/uav4geo/DroneDB/issues/12

1 Like