"Cannot process dataset"

I just purchased the Windows installer today (10 March 2023) to install on a hosted Windows Server (I needed the installer which runs natively on Windows, as the cloud provider, Paperspace, does not allow nested virtualization, which disqualifies Docker).

This is Windows WebODM v1.9.18 build 84.

I have a dataset (my very first one - I’m new to drone photogrammetry) which by now I have processed successfully both with Lightning’s free credits and with WebODM Docker on my own Windows laptop.

Download the dataset here:
Link 287 20MP JPG’s in a .zip. 2.77GB unzipped. All photos shot at minimum 75% overlap using DroneLink. Shot with DJI Air2S.

My next goal was to test more powerful machines to run WebODM on, as I’ll have even larger datasets to process in the near future.

Installing WebODM with the installer completed with no issues. I created a new project, added all 287 images to it, left the preset at Default, and turned image resizing off. After the images finish uploading to the node, processing begins but fails almost immediately with a “Cannot process dataset” error.

This error is occurring on a stock Windows Server 2022 Datacenter edition Paperspace Core virtual machine (a “C8” CPU instance, with a 16-core Xeon E5-2640v4 2.1GHz CPU and 60GB RAM). This server does not have a dedicated or integrated GPU (it’s using the “Microsoft Basic Display Adapter”).

I don’t suspect issues with this dataset, since as I mentioned I’ve already processed it successfully with WebODM twice before, using I believe the same build of WebODM (I did all installation and processing on 6 March 2023, so I used whatever version was current at that time). (On my laptop, I was able to process it only after I enabled image resizing, because 20GB of RAM, which was all I could spare Docker, wasn’t enough to process the full-resolution images). Lightning processed the full-resolution images just fine. I used the Default preset both times.

I grabbed copies of both the “app” and “pgsql” log files right after the error occurred, as well as the out-of-the-box node’s JSON config and can post them if needed. Task output is below:

Traceback (most recent call last):
File “C:\WebODM\resources\app\apps\ODM\run.py”, line 15, in
from opendm.utils import get_processing_results_paths, rm_r
File “C:\WebODM\resources\app\apps\ODM\opendm\utils.py”, line 5, in
from opendm.photo import find_largest_photo_dims
File “C:\WebODM\resources\app\apps\ODM\opendm\photo.py”, line 17, in
from opendm import get_image_size
File “C:\WebODM\resources\app\apps\ODM\opendm\get_image_size.py”, line 2, in
import cv2
File “C:\WebODM\resources\app\apps\ODM\venv\lib\site-packages\cv2_init_.py”, line 96, in
bootstrap()
File “C:\WebODM\resources\app\apps\ODM\venv\lib\site-packages\cv2_init_.py”, line 86, in bootstrap
import cv2
ImportError: DLL load failed while importing cv2: The specified module could not be found.

The log doesn’t indicate which DLL failed to load. Is there an easy way to tell?

1 Like

Some screenshots:

1 Like

1 Like

1 Like

1 Like

1 Like

1 Like

I downloaded and am processing your images now and all seems normal. I dont know what CV2 module is unfortunately, But I find it strange that your diagnostic picture isnt showing ram usage levels.

BTW welcome. This is a great supportive community and I imagine someone with more knowledge of this will respond before too long.

3 Likes

About the RAM usage chart - will that only appear when a job is actually processing? Admittedly I didn’t pay much attention to that. I can fire the machine back up to check whether that chart ever appears. If it only appears when a job is running, then I probably can’t take a screenshot fast enough before the job fails.

1 Like

Welcome!

We do not show the RAM for Windows native build at present.

As for cv2, that’s OpenCV, a core module we rely upon for functionality.

That being missing indicates that something interfered with the installation and blocked/removed that file, or is modifying the PATH.

Are you running an AV on that host? If so, allow-list the installer and the entire WebODM directory. Then, reboot the server and install in-place over the current installation of WebODM.

1 Like

Hi Saijin!

Thanks, that’s helpful. As far as I know, no AV is running (in fact, Paperspace appears to disable Windows Defender in their default image used to create this machine). I’ll try verifying that it’s disabled and will try a re-installation per your instructions.

Does the installer save a log anywhere? Could we use such a log to verify whether OpenCV was installed successfully?

1 Like

Defender should be fine to leave on, as that’s our only supported AV.

I do not believe we write out an install log, unfortunately.

Is this VM fresh, or is there a lot of other software installed already?

It’s as fresh as I can get from Paperspace. Here’s what’s installed out of the box:

1 Like

I was wrong - Windows Defender real-time protection actually wasn’t fully disabled before. I added exclusions for the software as well as the default install directory C:\WebODM, and then I completely disabled real-time protection. Then I re-installed WebODM, rebooted, deleted the job, re-created a fresh job, and the same thing occurred.

Then I tried completely uninstalling WebODM, rebooting, installing, and running another fresh job, and same thing.

Does anyone know which DLL is failing to load here?

1 Like

Can you please show the path entries for both system and user?

From what I can see, nothing should be interfering with installation and operation.

hopefully installation will get sorted out, because the dataset is nice and clean.

I did not realize Colorado had such non-mountainous regions.

2 Likes

If you’re not up and processing, please reach out to [email protected] so we can discuss remote support/troubleshooting options.