ODM stops processing with "cannot identify image file"

Hi all, I’ve been trying for hours to render the images from a ZENMUSE XT52 via WebODM using the multispectral option. I already process the original images and rename them (see files) and set in the XMP data the “BandName” and a “RigCameraIndex”. Meanwhile both images are used, but in the stage “export_visualsfm” the OpenSfm breaks (see logs).
I have already downloaded the images from the container. They exist and are readable, only Pillow does not get any further.

Does anyone have any ideas? Did I forget an option for Multispectral? I am very grateful for any advice!

Log files: ODM Failure - Pastebin.com
Processed images: Dropbox - ODM-FAILED.7z - Simplify your life

Greetings from Germany

2 Likes

Another try with everything default, before i set primary_band to RGB. Keeps crashing at the same line.

2021-12-04 04:54:28,961 DEBUG: Undistorting image 001_NIR.jpg
2021-12-04 04:54:29,034 DEBUG: Undistorting image 050_NIR.jpg
[INFO] running /code/SuperBuild/install/bin/opensfm/bin/opensfm export_visualsfm --points "/var/www/data/9cc630bc-a629-4372-9ed9-40ba1666280e/opensfm"
Traceback (most recent call last):
File "/code/SuperBuild/install/bin/opensfm/opensfm/io.py", line 1221, in image_size_from_fileobject
with Image.open(fb) as img:
File "/usr/local/lib/python3.9/dist-packages/PIL/Image.py", line 2943, in open
raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file <_io.BufferedReader name='/var/www/data/9cc630bc-a629-4372-9ed9-40ba1666280e/opensfm/undistorted/images/048_NIR.jpg.tif'>

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/code/SuperBuild/install/bin/opensfm/bin/opensfm_main.py", line 15, in
commands.command_runner(
File "/code/SuperBuild/install/bin/opensfm/opensfm/commands/command_runner.py", line 38, in command_runner
command.run(data, args)
File "/code/SuperBuild/install/bin/opensfm/opensfm/commands/command.py", line 12, in run
self.run_impl(data, args)
File "/code/SuperBuild/install/bin/opensfm/opensfm/commands/export_visualsfm.py", line 11, in run_impl
export_visualsfm.run_dataset(dataset, args.points, args.image_list)
File "/code/SuperBuild/install/bin/opensfm/opensfm/actions/export_visualsfm.py", line 29, in run_dataset
export(reconstructions[0], tracks_manager, udata, points, export_only)
File "/code/SuperBuild/install/bin/opensfm/opensfm/actions/export_visualsfm.py", line 49, in export
shot_size_cache[shot.id] = udata.undistorted_image_size(shot.id)
File "/code/SuperBuild/install/bin/opensfm/opensfm/dataset.py", line 1112, in undistorted_image_size
return self.io_handler.image_size(self._undistorted_image_file(image))
File "/code/SuperBuild/install/bin/opensfm/opensfm/io.py", line 1376, in image_size
return image_size_from_fileobject(fb)
File "/code/SuperBuild/install/bin/opensfm/opensfm/io.py", line 1226, in image_size_from_fileobject
image = imread(fb)
File "/code/SuperBuild/install/bin/opensfm/opensfm/io.py", line 1093, in imread
_, ext = os.path.splitext(path)
File "/usr/lib/python3.9/posixpath.py", line 118, in splitext
p = os.fspath(p)
TypeError: expected str, bytes or os.PathLike object, not BufferedReader

===== Dumping Info for Geeks (developers need this to fix bugs) =====
Child returned 1
Traceback (most recent call last):
File "/code/stages/odm_app.py", line 94, in execute
self.first_stage.run()
File "/code/opendm/types.py", line 346, in run
self.next_stage.run(outputs)
File "/code/opendm/types.py", line 346, in run
self.next_stage.run(outputs)
File "/code/opendm/types.py", line 346, in run
self.next_stage.run(outputs)
File "/code/opendm/types.py", line 327, in run
self.process(self.args, outputs)
File "/code/stages/run_opensfm.py", line 192, in process
octx.run('export_visualsfm --points')
File "/code/opendm/osfm.py", line 34, in run
system.run('%s %s "%s"' %
File "/code/opendm/system.py", line 106, in run
raise SubprocessException("Child returned {}".format(retcode), retcode)
opendm.system.SubprocessException: Child returned 1

===== Done, human-readable information to follow... =====

[ERROR] Uh oh! Processing stopped because of strange values in the reconstruction. This is often a sign that the input data has some issues or the software cannot deal with it. Have you followed best practices for data acquisition? See https://docs.opendronemap.org/flying/

Here is a dump of <task>/opensfm/undistorted/ Dropbox - odm-multispectral-test.zip - Simplify your life

1 Like

After searching a bit more i’ve found an open issue at ODM. Sadly there were no Updates

1 Like

Welcome!

Sorry for the trouble.

Are you able to drop the image 003_NIR from your fileset?

Also, how much space do you have available, and how much RAM?

Are you using the latest Docker images?

1 Like

Hey Saijin, thanks for your reply. I already tried that.

As seen in my second post i ran everything a second time (without dropping a file) and i got the exact same error but with a different file.

I’m running the latest docker image on my system. My system is running on an AMD Ryzen 5 3600X with 32GB of RAM. Renders with 350 4k images ran also successfully.

I investigated it further and installed ODM CLI tool. Same thing (so no Docker issue) but i was able to change the io library of opensfm where the error happens. It seems likre they try to use pillow first for PNG, JPG, etc. and if this does not work they fallback to rasterio. But inside the fallback they use an open buffered file reader as paramter - the fallback method expected a string object wich is a path object.
I moved the fallback method one method down and tada it worked.

def image_size(path):
    """Height and width of an image."""
    try:
        with open(path, "rb") as fb:
            return image_size_from_fileobject(fb)
    except Exception:
        # Slower fallback
        image = imread(path)
        return image.shape[:2]

It seems like opensfm changed their methods in the master branch but they also expect a string object and not a buffered reader.

This is the result (using fast-orthophoto) Dropbox - odm_orthophoto.tif - Simplify your life
Can you help me to evaluate the image? The thermal image is seen only very spotty. Is this because the camera isn’t supported?

1 Like

Try opening it in QGIS.

Raster “images” are data, not photographs like we’re used to. These especially are a different color bit depth and band arrangement, so they don’t normally draw correctly in regular image viewers.

1 Like

yh i did this ^^

1 Like

Ah, awesome! Glad to know you have QGIS installed. Very helpful :slight_smile:

Maybe adjusting the camera profile upstream in OpenSFM might help the reconstruction a bit…

That patchyness looks like we’re not getting equal reconstruction in each band, so there are some gaps in the bands, that when composited for view, lead to color shifts.

If you think about the rendered view as Raster Math of Band 1 (B) + Band 2 (G) + Band 3 (R), if any of the bands do not have sufficient data in a given spot, that math will change and the resultant color must therefore change.

Could you try bumping --feature-quality to ultra, and --min-num-features to something like 12000 or maybe 16000?

Hopefully we don’t Out-Of-Memory, so if you don’t have swap setup, it might be beneficial to add some (if you have the disk space).

have you looked at the thermal images? I hope 640x512 pixels are enough.

alright the task is running now with
--texturing-skip-global-seam-leveling --radiometric-calibration camera --pc-quality lowest --fast-orthophoto --feature-quality ultra --min-num-features 16000 using the cli so i don’t have a bottleneck with docker or something.
I guess combining everything with the --fast-orthophoto isn’t a problem?

btw. should i open a openfsm issue regarding the io.py file? :thinking:

2 Likes

i will hopefully be able to give some updates tonight (UTC). i Am not at home right now, but of course let everything continue to run :slight_smile:

1 Like

If you are able to open an issue or PR, that would be very much appreciated.

2 Likes

Alright this is the result using you suggested parameters. kinda better but more blueish.
Would be awesome if a developer could comment on this. Not when or how to render this exact dataset, but why the result looks like this. I would just like to be able to argue it to my colleagues. The idea was to use WebODM for our fire department, which we will definitely use with the HD images. Unfortunately, we will have to do without the thermal images with the Zenmuse.

Thanks for your help and also for this software!

1 Like

btw i opened a new PR at OpenSfM and tried to comment on the problem with a link to this post.

1 Like

Thanks for working to help us get this sorted!

Are you able to share the dataset (or other data from your Zenmuse) somewhere like dronedb.app?

It would help with testing and collecting information I need to add upstream OpenSFM profiles for it.

1 Like

for sure. here you go. Hub
if you need more information, more images or tests let me know!

2 Likes

Are you processing the color and thermal images separately?

That is the ideal flow for processing, especially when you’re dealing with visible (RGB) images with other data.

If you do process them separately, they end up stitching pretty well (minus a part of the roof that was too close for the tight FOV of the thermal camera).

1 Like

Oh wow you are a hero. I have not yet come up with the idea!
So you are processing the RGB images and thermal images seperatly to get two orthophotos. After all you create a new task with both orthophoto. Am i right?

I will test it for sure. We take more images at the end of this week - so more images to test.

Once again a huge thanks for this project!

1 Like

Yep!

You can keep them together in one “Project”, which is our highest level of organization in the WebODM UI.

Then you can process each dataset as a “Task”, which is the lowest level of organization.

A nice thing about grouping common Tasks together in a Project is that you can view the products all together in one view easily by using the “View Map” button for the Project.

Like this:

1 Like

As an aside, you might want to consider modifying your flight planning a bit if the Thermal images are a core product. Their FOV is much lower than the color imagery, so to ensure sufficient overlap/sidelap for thermal, you might have to increase overlap/sidelap or fly a bit higher.

Color:

Thermal:

You were mostly okay in this instance, but you could have had much better reconstruction over the roof if you were higher or had better overlap/sidelap.

2 Likes

Thanks for the PR in OpenSfM. I’ll try to include the patch in our fork at the next release :pray:

2 Likes