Processing video files with webODM?

OpenDroneMap Desktop
21

We have the new version of ODM running now so I’m testing the video processing.
In the mean time, a couple of questions:
(1) Have you tested in on 360 images? I the lens type is set correctly, will this more or less work or does one still need to use ODMax (aside from the additional control of frames, etc that ODMax provides)?

(2) How does the “video-resolution” setting interact with the “resize-images” setting?

(3) Does ODM by default use every frame in the video and how does this interact with the “video-limit” setting frame count? ie if one sets sets “video-limit” to say 500 on a 30s 30fps video (ie 900 frames) does ODM process every frame per second and stop when it gets to frame 500 or does it reduce the frame count accordingly over the whole length of the video?

Thanks in advance!

3 Likes
22

Hi Tim.

I am still very new to ODM, etc., but have dug into the video processing over recent days. I have a DJI Mini 2 and have been trying to feed ODM with old video captured previously when photogrammetry had not entered my head.

Today I’ve also been skimming through some of the source code.

Here are a few comments based on what I’ve seen/experienced so far:

  1. The main source code module for the video frame grabbing is video2dataset.py - and it is apparent it does not necessarily fully support every kind of video file you could throw at it. However it avoids blindly using every frame, instead filtering out frames that are 'too similar" and too blurred, etc.

  2. When using WebODM sometimes it won’t upload the input video. See Gordan’s earlier post. I thought it was some kind of size limit but that could be wrong.

  3. With my Mini 2 videos it is not extracting the GPS data that is in the subtitles stream. I haven’t had time to look close enough to see why yet. However if I use ffmpeg to manually extract the subtitles to a separate *.srt file then it picks up that information.

  4. Again, with the Mini 2 the meta data embedded in the subtitles is rather limited, both in scope (nothing about aircraft or gimbal orientation), and the GPS position is only given as decimal degrees with 4 decimals. That’s far less precise than usual so the positional data is “rough”. In addition to that, for the Mini 2 the information in subtitles is only updated once per second so that also potentially adds more noise to the position data if the drone is moving at any speed (although I did note the code appears to try to interpolate between position fixes.)

If it doesn’t pull out at least the GPS location data from the video then the rest of the processing is going to use different options that I suspect will only work in quite limited cases.

I have had some small successes but only after quite a lot of messing about, including clipping the original videos to include shorter segments that are more appropriate, etc. And now, literally in the last half hour, for several attempted video clip inputs it has given me the message as per Gordan’s first post in this thread.

So, at this stage, I wish you “good luck”!

4 Likes
23

Thanks for the additional info.

We got it running on our development branch this morning but that cluster is having GPU issues so I’ll have to wait until those are sorted before testing. I’ll report back.

For extracting gps… once you have extraced it, do you re-inject it back into the video in the correct format or are you using some other method to pass that data to ODM?
Also have you tried exiftools or does ffmpeg do more or less the same thing and seem sufficient? (on a somewhat related topic I got a new inst360 X3 and have been fighting with it to try and get usable GPS out).

2 Likes
24

One way to provide the GPS data is a *.srt file matching the basename of the video. So, along with DJI_0123.MP4 we could supply DJI_0123.srt and this is what I have done as a workaround.

The ODM code is pulling out the video frames it is going to use and after parsing the partner srt file injects the relevant information back into those frames so the rest of the processing can continue more or less "as per normal’.

Some DJI drones (maybe mainly earlier models?) generate the srt file alongside the video file. However with my drone (Mini 2) the subtitles are stored in the original video so I am using ffmpeg as a quick way to pull it out and generate the matching srt file. That extra step shouldn’t be necessary in theory but for some reason that I haven’t yet identified ODM is apparently failing to get the information direct from my videos.

If I don’t generate the srt file as described above then I get warnings about no GPS data plus subsequent effects from that.

2 Likes
25

I need to test locally, but another compatible implementation with external SRT with GPS and yaw/pitch/roll/heading will be OpenCamera on Android.

1 Like
26

My current workaround with failures to upload the video (WebODM, locally on Ubuntu with Docker) is to create the project/task, upload the SRT file (which I’ve generated first using ffmpeg) so the task folder and images folder exist and then use the command line to copy the actual video file to the appropriate docker location, and finally rerun.

After that things seem to run okay, all else being equal.

2 Likes
27

So an mp4 and an srt with a matching filename works in ODM? That’s great news!
I’ve been struggling to try and re-inject gps data back into my insta360 videos but haven’t yet succeeded so this would be a great solution (inst360 is a total dumpster fire when it comes to gps :exploding_head:)

3 Likes
28

@SeeNoEvil, can you expand a bit on the srt workflow? I’m trying to get insta360 X3 videos to work on ODM with GPS. Since the stupid insta360 processing software strips gps when converting raw 360 video to mp4, I’m using exiftools to create a GPX files from the raw video and then converting that to an SRT file and then I uploaded the output MP4 and the srt, with matching filenames, to ODM.
But the ODM Task output still finds no GPS.
There are a bunch of places this could be going wrong since I don’t rally know if there are variations in how GPX or SRT files are formatted and my gpx-<srt code was written by gpt4 :slight_smile:

Could you post a sample SRT file and.or the ffmpeg commands you are using to generate the srt file?

Thanks!

1 Like
29

Hi Tim.

Perhaps I can give you some actual samples later (data is stored on a different machine from what I have in front of me now) but from what you have said I think I can safely guess the insta360 X3 videos you are using will not have the GPS data in the (subtitle) format that the ODM tool is looking for. When I looked at the source code for that tool (a python script) it appears to only support a couple of variations that are in video generated by a number of DJI drones.

Because of the apparent limit on the video file size that be uploaded using WebODM I also generally first extracted a smaller section of the original video using ffmpeg (making sure to also copy the subtitle data when doing that) and then extracted the subtitle information from that smaller file to a separate SRT file in a separate step.

From memory, something like this:
ffmpeg -i original.mp4 -ss [start_time] -to [end_time] -vcodec copy -scodec copy part.mp4
ffmpeg -i part.mp4 part.srt

And then feed part.mp4 and part.srt into WedODM. (Or copy the mp4 file using docker command line method if it was still too big.)

On the issue of the size of the video, I also recently dug through the source code and found what appears to be a default limit of 256 MB, although I wasn’t 100% sure and there may also be a config file somewhere that allows this to be changed.

Seeing that I still have that page open in my browser, and just to get it recorded somewhere, the link to the search results is:

And in the results from that search there is a reference to app/static/app/js/vendor/dropzone.js which contains (on line 228): “maxFilesize: 256,” which appears to me to be the smoking gun.

The python code that ODM is using to grab frames from video and parse the SRT files can be found here:

Digging through that will show you the SRT formats that it currently accepts and other stuff that it does.

Finally, some related discussion:

1 Like
30

@Tim

I’ve just corrected an error in the ffmpeg command options in my post above. I wrote -vcopy -scopy originally instead of -vcodec copy -scodec copy (to copy the video codec and subtitle stuff from the input file).

Below is 10 seconds of a *.SRT file where the original data is from a DJI Mini 2. You will see this has latitude and longitude in decimal degrees to four decimal places only. You may be able to output higher precision values. Also the Mini 2 only updates the subtitle info once per second so that adds to the inaccuracy of the GPS data. The video2dataset.py script that ODM is using attempts to interpolate but you may have source data that is more timely. Look at the source code module srtparser.py for the other slightly different format it can also handle.

1
00:00:00,000 --> 00:00:01,000
F/2.8, SS 949.23, ISO 110, EV 0, DZOOM 1.000, GPS (175.4009, -39.1127, 21), D 0.54m, H 5.70m, H.S 0.36m/s, V.S -0.00m/s 


2
00:00:01,000 --> 00:00:02,000
F/2.8, SS 949.23, ISO 110, EV 0, DZOOM 1.000, GPS (175.4010, -39.1127, 21), D 1.17m, H 5.80m, H.S 0.81m/s, V.S -0.00m/s 


3
00:00:02,000 --> 00:00:03,000
F/2.8, SS 949.23, ISO 110, EV 0, DZOOM 1.000, GPS (175.4010, -39.1127, 21), D 2.43m, H 5.80m, H.S 1.34m/s, V.S -0.00m/s 


4
00:00:03,000 --> 00:00:04,000
F/2.8, SS 949.23, ISO 110, EV 0, DZOOM 1.000, GPS (175.4010, -39.1127, 21), D 3.90m, H 5.70m, H.S 1.49m/s, V.S -0.00m/s 


5
00:00:04,000 --> 00:00:05,000
F/2.8, SS 944.11, ISO 110, EV 0, DZOOM 1.000, GPS (175.4010, -39.1127, 21), D 5.66m, H 5.80m, H.S 1.58m/s, V.S -0.00m/s 


6
00:00:05,000 --> 00:00:06,000
F/2.8, SS 928.89, ISO 110, EV 0, DZOOM 1.000, GPS (175.4010, -39.1127, 21), D 7.34m, H 5.50m, H.S 1.62m/s, V.S -0.00m/s 


7
00:00:06,000 --> 00:00:07,000
F/2.8, SS 939.01, ISO 110, EV 0, DZOOM 1.000, GPS (175.4010, -39.1127, 21), D 9.02m, H 5.40m, H.S 1.58m/s, V.S -0.00m/s 


8
00:00:07,000 --> 00:00:08,000
F/2.8, SS 985.91, ISO 110, EV 0, DZOOM 1.000, GPS (175.4010, -39.1127, 21), D 10.46m, H 5.20m, H.S 1.26m/s, V.S -0.00m/s 


9
00:00:08,000 --> 00:00:09,000
F/2.8, SS 1002.05, ISO 110, EV 0, DZOOM 1.000, GPS (175.4010, -39.1127, 21), D 11.70m, H 5.20m, H.S 1.14m/s, V.S -0.00m/s 


10
00:00:09,000 --> 00:00:10,000
F/2.8, SS 1035.15, ISO 110, EV 0, DZOOM 1.000, GPS (175.4010, -39.1126, 21), D 12.79m, H 5.20m, H.S 1.12m/s, V.S -0.00m/s
1 Like
31

That all is super helpful. Thanks so much!

1 Like
32

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.