ODM stops processing with "cannot identify image file"

Gurkengewuerz · 4 December 2021 04:40

Hi all, I’ve been trying for hours to render the images from a ZENMUSE XT52 via WebODM using the multispectral option. I already process the original images and rename them (see files) and set in the XMP data the “BandName” and a “RigCameraIndex”. Meanwhile both images are used, but in the stage “export_visualsfm” the OpenSfm breaks (see logs).
I have already downloaded the images from the container. They exist and are readable, only Pillow does not get any further.

Does anyone have any ideas? Did I forget an option for Multispectral? I am very grateful for any advice!

Log files: ODM Failure - Pastebin.com
Processed images: Dropbox - ODM-FAILED.7z - Simplify your life

Greetings from Germany

Gurkengewuerz · 4 December 2021 05:21

Another try with everything default, before i set primary_band to RGB. Keeps crashing at the same line.

2021-12-04 04:54:28,961 DEBUG: Undistorting image 001_NIR.jpg
2021-12-04 04:54:29,034 DEBUG: Undistorting image 050_NIR.jpg
[INFO] running /code/SuperBuild/install/bin/opensfm/bin/opensfm export_visualsfm --points "/var/www/data/9cc630bc-a629-4372-9ed9-40ba1666280e/opensfm"
Traceback (most recent call last):
File "/code/SuperBuild/install/bin/opensfm/opensfm/io.py", line 1221, in image_size_from_fileobject
with Image.open(fb) as img:
File "/usr/local/lib/python3.9/dist-packages/PIL/Image.py", line 2943, in open
raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file <_io.BufferedReader name='/var/www/data/9cc630bc-a629-4372-9ed9-40ba1666280e/opensfm/undistorted/images/048_NIR.jpg.tif'>

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/code/SuperBuild/install/bin/opensfm/bin/opensfm_main.py", line 15, in
commands.command_runner(
File "/code/SuperBuild/install/bin/opensfm/opensfm/commands/command_runner.py", line 38, in command_runner
command.run(data, args)
File "/code/SuperBuild/install/bin/opensfm/opensfm/commands/command.py", line 12, in run
self.run_impl(data, args)
File "/code/SuperBuild/install/bin/opensfm/opensfm/commands/export_visualsfm.py", line 11, in run_impl
export_visualsfm.run_dataset(dataset, args.points, args.image_list)
File "/code/SuperBuild/install/bin/opensfm/opensfm/actions/export_visualsfm.py", line 29, in run_dataset
export(reconstructions[0], tracks_manager, udata, points, export_only)
File "/code/SuperBuild/install/bin/opensfm/opensfm/actions/export_visualsfm.py", line 49, in export
shot_size_cache[shot.id] = udata.undistorted_image_size(shot.id)
File "/code/SuperBuild/install/bin/opensfm/opensfm/dataset.py", line 1112, in undistorted_image_size
return self.io_handler.image_size(self._undistorted_image_file(image))
File "/code/SuperBuild/install/bin/opensfm/opensfm/io.py", line 1376, in image_size
return image_size_from_fileobject(fb)
File "/code/SuperBuild/install/bin/opensfm/opensfm/io.py", line 1226, in image_size_from_fileobject
image = imread(fb)
File "/code/SuperBuild/install/bin/opensfm/opensfm/io.py", line 1093, in imread
_, ext = os.path.splitext(path)
File "/usr/lib/python3.9/posixpath.py", line 118, in splitext
p = os.fspath(p)
TypeError: expected str, bytes or os.PathLike object, not BufferedReader

===== Dumping Info for Geeks (developers need this to fix bugs) =====
Child returned 1
Traceback (most recent call last):
File "/code/stages/odm_app.py", line 94, in execute
self.first_stage.run()
File "/code/opendm/types.py", line 346, in run
self.next_stage.run(outputs)
File "/code/opendm/types.py", line 346, in run
self.next_stage.run(outputs)
File "/code/opendm/types.py", line 346, in run
self.next_stage.run(outputs)
File "/code/opendm/types.py", line 327, in run
self.process(self.args, outputs)
File "/code/stages/run_opensfm.py", line 192, in process
octx.run('export_visualsfm --points')
File "/code/opendm/osfm.py", line 34, in run
system.run('%s %s "%s"' %
File "/code/opendm/system.py", line 106, in run
raise SubprocessException("Child returned {}".format(retcode), retcode)
opendm.system.SubprocessException: Child returned 1

===== Done, human-readable information to follow... =====

[ERROR] Uh oh! Processing stopped because of strange values in the reconstruction. This is often a sign that the input data has some issues or the software cannot deal with it. Have you followed best practices for data acquisition? See https://docs.opendronemap.org/flying/

Here is a dump of <task>/opensfm/undistorted/ Dropbox - odm-multispectral-test.zip - Simplify your life

Gurkengewuerz · 4 December 2021 13:48

After searching a bit more i’ve found an open issue at ODM. Sadly there were no Updates

github.com/OpenDroneMap/ODM

radiometric-calibration doesn't work properly

opened 06:16AM - 30 Oct 21 UTC

lucadelu

help wanted possible bug

### How did you install OpenDroneMap? (Docker, natively, ...)? Docker using w…ebodm on linux server ### What's your browser and operating system? (Copy/paste the output of https://www.whatismybrowser.com/) Firefox/Chromium on Linux ### What is the problem? Using Multispectral images coming from Mapir Camera and choosing `radiometric-calibration: camera` I'm not able to process the data. The problem seems during the opensfm stage because it is looking for a .tif file but instead it is a .JPG image ~~~ [INFO] running /code/SuperBuild/install/bin/opensfm/bin/opensfm export_visualsfm --points "/var/www/data/34a4d273-e578-49ca-a7fe-3feda8bc6217/opensfm" Traceback (most recent call last): File "/code/SuperBuild/install/bin/opensfm/opensfm/io.py", line 1221, in image_size_from_fileobject with Image.open(fb) as img: File "/usr/local/lib/python3.9/dist-packages/PIL/Image.py", line 2943, in open raise UnidentifiedImageError( PIL.UnidentifiedImageError: cannot identify image file <_io.BufferedReader name='/var/www/data/34a4d273-e578-49ca-a7fe-3feda8bc6217/opensfm/undistorted/images/2021_0922_145250_210.JPG.tif'> ~~~ ### What should be the expected behavior? If this is a feature request, please describe in detail the changes you think should be made to the code, citing files and lines where changes should be made, if possible. Finishing the procedure for the reconstruction ### How can we reproduce this? (What steps did you do to trigger the problem? What parameters are you using for processing? If possible please include a copy of your dataset uploaded on Google Drive or Dropbox. Be detailed) You can try with this dataset https://drive.google.com/file/d/1sa-P_VzhsuGrAw4gb6n5ePU5qhDqm-zj/view?usp=sharing I also attach the full log [console.txt](https://github.com/OpenDroneMap/ODM/files/7445937/console.txt)

Saijin_Naib · 4 December 2021 15:12

Welcome!

Sorry for the trouble.

Are you able to drop the image 003_NIR from your fileset?

Also, how much space do you have available, and how much RAM?

Are you using the latest Docker images?

Gurkengewuerz · 4 December 2021 15:25

Hey Saijin, thanks for your reply. I already tried that.

As seen in my second post i ran everything a second time (without dropping a file) and i got the exact same error but with a different file.

I’m running the latest docker image on my system. My system is running on an AMD Ryzen 5 3600X with 32GB of RAM. Renders with 350 4k images ran also successfully.

I investigated it further and installed ODM CLI tool. Same thing (so no Docker issue) but i was able to change the io library of opensfm where the error happens. It seems likre they try to use pillow first for PNG, JPG, etc. and if this does not work they fallback to rasterio. But inside the fallback they use an open buffered file reader as paramter - the fallback method expected a string object wich is a path object.
I moved the fallback method one method down and tada it worked.

def image_size(path):
    """Height and width of an image."""
    try:
        with open(path, "rb") as fb:
            return image_size_from_fileobject(fb)
    except Exception:
        # Slower fallback
        image = imread(path)
        return image.shape[:2]

It seems like opensfm changed their methods in the master branch but they also expect a string object and not a buffered reader.

github.com

mapillary/OpenSfM/blob/main/opensfm/io.py#L1170-L1174

    
      
          def imread(
              path: str, grayscale: bool = False, unchanged: bool = False, anydepth: bool = False
          ):
              with open(path, "rb") as fb:
                  return imread_from_fileobject(fb, grayscale, unchanged, anydepth)

This is the result (using fast-orthophoto) Dropbox - odm_orthophoto.tif - Simplify your life
Can you help me to evaluate the image? The thermal image is seen only very spotty. Is this because the camera isn’t supported?

Saijin_Naib · 4 December 2021 15:49

Try opening it in QGIS.

Raster “images” are data, not photographs like we’re used to. These especially are a different color bit depth and band arrangement, so they don’t normally draw correctly in regular image viewers.

Gurkengewuerz · 4 December 2021 15:51

yh i did this ^^

Saijin_Naib · 4 December 2021 15:54

Ah, awesome! Glad to know you have QGIS installed. Very helpful

Maybe adjusting the camera profile upstream in OpenSFM might help the reconstruction a bit…

That patchyness looks like we’re not getting equal reconstruction in each band, so there are some gaps in the bands, that when composited for view, lead to color shifts.

If you think about the rendered view as Raster Math of Band 1 (B) + Band 2 (G) + Band 3 (R), if any of the bands do not have sufficient data in a given spot, that math will change and the resultant color must therefore change.

Could you try bumping --feature-quality to ultra, and --min-num-features to something like 12000 or maybe 16000?

Hopefully we don’t Out-Of-Memory, so if you don’t have swap setup, it might be beneficial to add some (if you have the disk space).

Gurkengewuerz · 4 December 2021 16:00

have you looked at the thermal images? I hope 640x512 pixels are enough.

alright the task is running now with
--texturing-skip-global-seam-leveling --radiometric-calibration camera --pc-quality lowest --fast-orthophoto --feature-quality ultra --min-num-features 16000 using the cli so i don’t have a bottleneck with docker or something.
I guess combining everything with the --fast-orthophoto isn’t a problem?

btw. should i open a openfsm issue regarding the io.py file?

Gurkengewuerz · 4 December 2021 16:19

i will hopefully be able to give some updates tonight (UTC). i Am not at home right now, but of course let everything continue to run

Saijin_Naib · 4 December 2021 16:39

If you are able to open an issue or PR, that would be very much appreciated.

Gurkengewuerz · 4 December 2021 20:42

Alright this is the result using you suggested parameters. kinda better but more blueish.
Would be awesome if a developer could comment on this. Not when or how to render this exact dataset, but why the result looks like this. I would just like to be able to argue it to my colleagues. The idea was to use WebODM for our fire department, which we will definitely use with the HD images. Unfortunately, we will have to do without the thermal images with the Zenmuse.

Thanks for your help and also for this software!

Gurkengewuerz · 4 December 2021 20:58

btw i opened a new PR at OpenSfM and tried to comment on the problem with a link to this post.

github.com/mapillary/OpenSfM

fixed fallback method for image size

mapillary:main ← Gurkengewuerz:io-path-fix

opened 08:55PM - 04 Dec 21 UTC

Gurkengewuerz

+1 -1

Hello all. While using OpenDroneMap/ODM I noticed a presumable bug in OpenSfM. C…orrect me if I am wrong here! The following issue has appeared. ODM has called the export_visualsfm command. OpenSfM has tried to open tif images via the method image_size_from_fileobject. Since Pillow doesn't support tif images the slower fallback method via imread is used. The problem is that image_size_from_fileobject gets a file object which is passed one to one in imread. imread expects a path! Therefore my PR to pass the path of the opened file. To understand the ODM problem more: https://community.opendronemap.org/t/odm-stops-processing-with-cannot-identify-image-file/9739

Saijin_Naib · 4 December 2021 23:24

Thanks for working to help us get this sorted!

Are you able to share the dataset (or other data from your Zenmuse) somewhere like dronedb.app?

It would help with testing and collecting information I need to add upstream OpenSFM profiles for it.

Gurkengewuerz · 4 December 2021 23:51

for sure. here you go. Hub
if you need more information, more images or tests let me know!

Saijin_Naib · 6 December 2021 16:30

Are you processing the color and thermal images separately?

That is the ideal flow for processing, especially when you’re dealing with visible (RGB) images with other data.

If you do process them separately, they end up stitching pretty well (minus a part of the roof that was too close for the tight FOV of the thermal camera).

Gurkengewuerz · 6 December 2021 16:48

Oh wow you are a hero. I have not yet come up with the idea!
So you are processing the RGB images and thermal images seperatly to get two orthophotos. After all you create a new task with both orthophoto. Am i right?

I will test it for sure. We take more images at the end of this week - so more images to test.

Once again a huge thanks for this project!

Saijin_Naib · 6 December 2021 17:30

Yep!

You can keep them together in one “Project”, which is our highest level of organization in the WebODM UI.

Then you can process each dataset as a “Task”, which is the lowest level of organization.

A nice thing about grouping common Tasks together in a Project is that you can view the products all together in one view easily by using the “View Map” button for the Project.

Like this:

Saijin_Naib · 6 December 2021 17:37

As an aside, you might want to consider modifying your flight planning a bit if the Thermal images are a core product. Their FOV is much lower than the color imagery, so to ensure sufficient overlap/sidelap for thermal, you might have to increase overlap/sidelap or fly a bit higher.

Color:

Thermal:

You were mostly okay in this instance, but you could have had much better reconstruction over the roof if you were higher or had better overlap/sidelap.

pierotofy · 6 December 2021 18:06

Thanks for the PR in OpenSfM. I’ll try to include the patch in our fork at the next release