Error - Cannot process dataset

Hello everyone.
I have a study area that I had successfully used WebODM got the NIR orthophoto in May. So I used the same flight planning by DJI GS Pro app, same drone (DJI Phantom4 pro) and NIR camera (MAPIR Survey3N), and then collected images data in July. However, I cannot get NIR orthophoto by using the new images dataset of July, and here always had “Cannot process dataset” error.

My images dataset:
https://drive.google.com/drive/folders/12NwObdY-Ug7yg4hSdlWz0WzmuheIkFtn?usp=sharing

The entire task output: (since there is a limitation on number of words in the Topic, so I copy task output in a google docs)

The parameters that setup for processing:
High Resolution, and resize images: yes, 2048px
So under my task it shows task details as:
Processing Node: node-odm-1 (manual)
Options: ignore-gsd: true, dsm: true, pc-quality: high, dem-resolution: 2.0, orthophoto-resolution: 2.0

The browser I’m using is Safari of macOS Big Sur version 11.3.1. (Processor:2 GHz Quad-Core Intel Core i5; Memory: 16 GB 3733 MHz LPDDR4X). And I manually installed the WebODM on my computer.

I would appreciate your kind help.

1 Like

Welcome!

Sorry to hear you’re having issues processing that second collection.

Have you tried processing it with Resize Images disabled? It can have a potentially negative effect on the Tie Point extraction, which can lead to failed reconstruction.

1 Like

Thank you for your reply.
I just tried without resize images, but it still didn’t work with the “Cannot process dataset” error…

Do you think maybe there is an issue in my image dataset? I just guess such as images are not continuous enough (but in the flight planning I set the overlap ratio is 90%) or the sunlight is not enough (but I can got the RGB orthophoto through the RGB images taken by the DJI drone) or the flight hight should decrease? Any other comments or suggestions will be appreciated.

1 Like

I’m trying it now with some pre-processing in XnConvert.

Overlap sounds good at 90%. Is sidelap also 90%?

Yes, scene brightness is a bit low, but not terrible.

Ideally, you’d INCREASE flight height to have a better chance of finding tiepoints, not decrease.

I’ll let you know what happens once this processes.

2 Likes

The dataset images actually were pre-process by MAPIR MCC software. The raw data of NIR have two types of file, one is .RAW, the other is .JPG. After process through the MCC software, I got the image dataset that in the google drive. I was also thinking whether this step cause the error…

The front overlap ratio is 90% and side overlap ratio is 95%.

Yeah, your comments reminder me that actually in May, I also tried several times to get the orthophoto, and the successful orthophoto was got in flight height is 115ft instead of 70ft before. At that time, I read some where said that the flight height should be 3-4 times higher than the highest objects of the study area. So when I successfully got orthophoto with 115ft, I thought that I found the right height to do my study, since I want to keep tracking the study area change with the time under same setup.

Look forward for your pre-processing results. Hope it’s a good results. Thank you!

2 Likes

Is your pre-processing writing out the geoinformation to the EXIF tags for the TIFF images? It does not appear to be there in the dataset you uploaded.

Are you able to upload the original RAW+JPG files as well?

2 Likes

Honestly I don’t know the answer of your first question, so I just wrote an email to ask about it. I don’t get reply yet, but what I know is these two types of file that are captured at the same time since the pixel information is stored in the RAW and the metadata is stored in the JPG. Additionaly, there is an individual GPS connected with MAPIR camera, and every time before I collected the image data, I made sure that GPS connected well.

Here is the original RAW+JPG files:
https://drive.google.com/drive/folders/1T8BjgVV3ZL4O8_SMTXiQqyxv_QkXq9UY?usp=sharing
Thank you.

1 Like


So, I think it might have to do with two of your images having completely insane GPS values, and making the processing extent unsolvable.

157.TIF and 167.TIF (158.JPG and 168.JPG in the original/RAW) are showing up across the ocean off the Eastern coast of Spain.

I’m re-running with these erroneous images removed now:


jixue1-Report.pdf (6.1 MB)

2 Likes

Thank you very much for your explanation, help to find the root cause and summarizing the report.

Please let me confirm about the root cause and make sure that I understand well. So, the GPS values of the part of image dataset are wrong. Without the wrong GPS values of images, the “cannot process dataset” issue can be solved. Is it right?

So could you please explain or provide some more comments on this root cause based on your rich experience? Like why occurred the total wrong GPS values of the images? Is it caused by the individual GPS which connected with MAPIR camera? or is it caused by the pre-processing MCC software? In addition, I saw you used some tool to check the geotagging of images. Is this a regular process need to be done before processing a dataset to guarantee image process successfully?

I’m sorry that I’m asking a lot of question. But it will really help me a lot. Thank you!

1 Like

Yes, two images had very wrong longitude values, putting them half a world away. This makes the reconstruction extent extremely large.

Fixing these values, or removing the bad images, is all that was required.

I would guess faulty GPS and/or buggy firmware causing the wrong values to be written to the images. I don’t think it was their pre-processing software as the original images (RAW+JPG) dataset also has this problem.

I don’t normally pre-check image locations, but I will from now on.

I opened a Feature Request here:

2 Likes

Thank you very much for your comments. I think it will very important and helpful if there is a geolocation check of images before processing the dataset as described in your request.

Before they approve your request, may I ask about the software that you used to check the geolocation of the images? Is it called “Geotagging”? Is it free to use? Since I’m thinking to use it by myself to check my other datasets that were collected in June, they may have the same issue.

1 Like

If you think my Feature Request will help, please feel free to leave a comment on it for Piero.

Both softwares I used were in the Flathub repository for Linux. The names are as I listed. I’m not sure if they are available for Windows or Mac.

You can also use QGIS with the Import Geotagged Photos processing algorithm to make your own quick check map.

1 Like

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.