Error processing stacked tiff output from MicaSense imageprocessing

and-viceversa · 31 March 2021 19:02

Running WebODM on Windows 10 with Docker, using Microsoft Edge and Firefox.

Continuing the discussion about processing MicaSense imagery here and here. Especially pierotofy’s comment.

Attempted workflow:
Raw imagery > MicaSense imageprocessing > band stack tif > WebODM > orthomosaic

Output test images from MicaSense imageprocessing are here.

Using the Default processing mode yields the error:

[INFO]    Initializing ODM - Wed Mar 31 18:04:05  2021
[INFO]    ==============
[INFO]    build_overviews: False
[INFO]    camera_lens: auto
[INFO]    cameras: {}
[INFO]    crop: 3
[INFO]    debug: False
[INFO]    dem_decimation: 1
[INFO]    dem_euclidean_map: False
[INFO]    dem_gapfill_steps: 3
[INFO]    dem_resolution: 5
[INFO]    depthmap_resolution: 640
[INFO]    dsm: True
[INFO]    dtm: False
[INFO]    end_with: odm_report
[INFO]    fast_orthophoto: False
[INFO]    feature_quality: high
[INFO]    feature_type: sift
[INFO]    force_gps: False
[INFO]    gcp: None
[INFO]    geo: None
[INFO]    gps_accuracy: 10
[INFO]    ignore_gsd: False
[INFO]    matcher_distance: 0
[INFO]    matcher_neighbors: 8
[INFO]    matcher_type: flann
[INFO]    max_concurrency: 32
[INFO]    merge: all
[INFO]    mesh_octree_depth: 11
[INFO]    mesh_size: 200000
[INFO]    min_num_features: 8000
[INFO]    name: 5d2eac27-1d89-43b9-9f08-ca38cbc1dde6
[INFO]    optimize_disk_space: False
[INFO]    orthophoto_compression: DEFLATE
[INFO]    orthophoto_cutline: False
[INFO]    orthophoto_no_tiled: False
[INFO]    orthophoto_png: False
[INFO]    orthophoto_resolution: 5
[INFO]    pc_classify: False
[INFO]    pc_csv: False
[INFO]    pc_ept: False
[INFO]    pc_filter: 2.5
[INFO]    pc_las: False
[INFO]    pc_quality: medium
[INFO]    pc_rectify: False
[INFO]    pc_sample: 0
[INFO]    pc_tile: False
[INFO]    primary_band: auto
[INFO]    project_path: /var/www/data
[INFO]    radiometric_calibration: none
[INFO]    rerun: None
[INFO]    rerun_all: False
[INFO]    rerun_from: None
[INFO]    resize_to: 2048
[INFO]    skip_3dmodel: False
[INFO]    skip_band_alignment: False
[INFO]    skip_report: False
[INFO]    sm_cluster: None
[INFO]    smrf_scalar: 1.25
[INFO]    smrf_slope: 0.15
[INFO]    smrf_threshold: 0.5
[INFO]    smrf_window: 18.0
[INFO]    split: 999999
[INFO]    split_image_groups: None
[INFO]    split_overlap: 150
[INFO]    texturing_data_term: gmi
[INFO]    texturing_outlier_removal_type: gauss_clamping
[INFO]    texturing_skip_global_seam_leveling: False
[INFO]    texturing_skip_local_seam_leveling: False
[INFO]    texturing_tone_mapping: none
[INFO]    tiles: False
[INFO]    time: False
[INFO]    use_3dmesh: False
[INFO]    use_exif: False
[INFO]    use_fixed_camera_params: False
[INFO]    use_hybrid_bundle_adjustment: False
[INFO]    verbose: False
[INFO]    ==============
[INFO]    Running dataset stage
[INFO]    Loading dataset from: /var/www/data/5d2eac27-1d89-43b9-9f08-ca38cbc1dde6/images
[INFO]    Loading 123 images
[WARNING] Cannot read /var/www/data/5d2eac27-1d89-43b9-9f08-ca38cbc1dde6/images/PH4Rh4OMhUC0J8to3r0J.tif with PIL, fallback to cv2: cannot identify image file '/var/www/data/5d2eac27-1d89-43b9-9f08-ca38cbc1dde6/images/PH4Rh4OMhUC0J8to3r0J.tif'
imread_('/var/www/data/5d2eac27-1d89-43b9-9f08-ca38cbc1dde6/images/PH4Rh4OMhUC0J8to3r0J.tif'): can't read header: OpenCV(4.5.0) /code/SuperBuild/src/opencv/modules/imgcodecs/src/grfmt_tiff.cpp:148: error: (-215:Assertion failed) channels <= 4 in function 'normalizeChannelsNumber'
Traceback (most recent call last):
File "/code/opendm/get_image_size.py", line 13, in get_image_size
with Image.open(file_path) as img:
File "/usr/local/lib/python3.8/dist-packages/PIL/Image.py", line 2943, in open
raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file '/var/www/data/5d2eac27-1d89-43b9-9f08-ca38cbc1dde6/images/PH4Rh4OMhUC0J8to3r0J.tif'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/code/run.py", line 68, in <module>
app.execute()
File "/code/stages/odm_app.py", line 81, in execute
self.first_stage.run()
File "/code/opendm/types.py", line 319, in run
self.process(self.args, outputs)
File "/code/stages/dataset.py", line 109, in process
p = types.ODM_Photo(f)
File "/code/opendm/photo.py", line 80, in __init__
self.parse_exif_values(path_file)
File "/code/opendm/photo.py", line 260, in parse_exif_values
self.width, self.height = get_image_size.get_image_size(_path_file)
File "/code/opendm/get_image_size.py", line 19, in get_image_size
width = img.shape[1]
AttributeError: 'NoneType' object has no attribute 'shape'

It looks like ODM is having a difficult time opening the band stack tifs output from MicaSense imageprocessing. The same occurs when I skip-band-alignment.

Here’s some relevant MicaSense code where the tifs are written (with GDAL):

driver = GetDriverByName('GTiff')

out_raster = driver.Create(outfile_name, cols, rows, bands, GDT_UInt16,
                                   options=['INTERLEAVE=BAND', 'COMPRESS=DEFLATE', f'PHOTOMETRIC={photometric}'])

Any guesses?

Saijin_Naib · 1 April 2021 20:03

Hmm, it looks like one of the libraries we use isn’t properly handling the multi-band TIFFs when they’re more than 4 bands.

Your dataset is 5 bands.

EDIT:
Don’t think it is PIL, looks like that can handle N-number of bands properly (as it should be able to):
https://pillow.readthedocs.io/en/stable/handbook/concepts.html?highlight=band#concepts

and-viceversa · 7 April 2021 00:22

As a place holder, here are some links with similar issues.

github.com/python-pillow/Pillow

Tracking Issue for high bit depth multichannel images

opened 04:09PM - 05 May 16 UTC

wiredfool

Enhancement NumPy

Pillow (and PIL) is currently able to open 8 bit per channel multi-channel image…s (such as RGB) but is able to open higher bit depth images (e.g. I16, I32, or Float32 images) if they are single channel (e.g., grayscale). ## Previous References This has been requested many times: #1828, #1885, #1839, #1602, and farther back. ## Requirements - We should be able to support common GIS formats as well as high bit depth RGB(A) images. - At least 4 channels, but potentially more (see #1839) - Different pixel formats, including I16, I32, and Float. - There should be definitions for the array interface to exchange images with numpy/scipy - There should be enough support to read and write TIFFs and raw image data. - Support for resize, crop, and convert operations at the very least. ## Background Reference Info The rough sequence for image loading is: - Image file is opened - Each of the ImagePlugin _accept functions have a chance to look at the first few bytes to determine if they should attempt to open the file - The `*ImagePlugin._open` method is called giving the image plugin a chance to read more of the image and determine if it still wants to consider it a valid image of it's particular type. If it does, it passes back a tile definition which includes a decoder and an image size. - If there is a successful _open call, at some point later `*ImagePlugin._load` may be called on the image, which runs the decoder producing a set of bytes in a raw mode. This is where things like compression are handled, but the output of the decoder is not necessarily what we're storing in our internal structures. - The image is unpacked (`Unpack.c`) from the raw mode (e.g. `I16;BS`) into a storage (`Storage.c`) mode (`I`). - It's now possible to operate on the image (e.g. crop, pixel access, etc) There are 3 (or 4) image data pointers, as defined in Imaging.h: ``` struct ImagingMemoryInstance { /* Format */ char mode[IMAGING_MODE_LENGTH]; /* Band names ("1", "L", "P", "RGB", "RGBA", "CMYK", "YCbCr", "BGR;xy") */ int type; /* Data type (IMAGING_TYPE_*) */ int depth; /* Depth (ignored in this version) */ int bands; /* Number of bands (1, 2, 3, or 4) */ int xsize; /* Image dimension. */ int ysize; /* Colour palette (for "P" images only) */ ImagingPalette palette; /* Data pointers */ UINT8 **image8; /* Set for 8-bit images (pixelsize=1). */ INT32 **image32; /* Set for 32-bit images (pixelsize=4). */ /* Internals */ char **image; /* Actual raster data. */ char *block; /* Set if data is allocated in a single block. */ int pixelsize; /* Size of a pixel, in bytes (1, 2 or 4) */ int linesize; /* Size of a line, in bytes (xsize * pixelsize) */ /* Virtual methods */ void (*destroy)(Imaging im); }; ``` The only one that is guaranteed to be set is `**image`, which is an array of pointers to row data. ## Changes Required - Definitions for all of the modes that we're planning, and potentially a [format];MB[#bands] style generic mode. ### Core Imaging Structure - The imaging structure has the fields required to add the additional channels. (type, bands, pixelsize, linesize) - The `**image` pointer can be used for any width of pixel. - We may or may not want to set the `**image32` pointer. - Currently type of `IMAGING_TYPE_INT32` and `IMAGING_TYPE_FLOAT32` imply 1 band. This will change. - Consider promoting int16 to `IMAGING_TYPE_INT16` ### Storage - Updates to `Storage.c`, `Unpack.c`, `Pack.c`, `Access.c`, `PyAccess.py`, and `Convert.c` ## Ways to Help We need a better definition of the format requirements. What are the various types of images that are used in GIS, Medical, or other fields that we'd want to interpret? We need small, redistributable versions of images that we can test against. [in progress]

(Still open since 2016!)

Here’s a snippet using PIL, cv2, rasterio, and GDAL. rasterio and GDAL succeed. Not really sure what’s going on here because this is a common use case.

# PIL Fail
im = Image.open(img_one)
im.load()

# OpenCV Fail
im = cv2.imread(img_one)

# rasterio Success
with rasterio.open(img_one) as f:
    print(f.count)

# GDAL Success
im = gdal.Open(img_one).ReadAsArray()
print(im)

Still looking into this…

Edit: Based on the open issue above, it’s looking like PIL cannot handle images with more bands than 4 (RGBA), even though it can handle a diverse range of bit-depths.

and-viceversa · 7 April 2021 01:32

Looks like regardless of the initial issue, ODM does not currently support multi band 16bit TIFFs.

github.com/OpenDroneMap/ODM

Add support for multi-layer 16bit TIFFs

opened 07:37AM - 27 Feb 21 UTC

YungKC

bug enhancement help wanted

### How did you install OpenDroneMap? (Docker, natively, ...)? Docker ### …What's your browser and operating system? (Copy/paste the output of https://www.whatismybrowser.com/) Chrome 88 on Windows 10 ### What is the problem? OpenMVS causes ODM to crash while trying to process TIFF files ### What should be the expected behavior? If this is a feature request, please describe in detail the changes you think should be made to the code, citing files and lines where changes should be made, if possible. Should not crash. The original JPG file processed as expected. ### How can we reproduce this? (What steps did you do to trigger the problem? What parameters are you using for processing? If possible please include a copy of your dataset uploaded on Google Drive or Dropbox. Be detailed) This was referenced by another user before: https://community.opendronemap.org/t/error-processing-tiff-images/6276 I experienced the same error message at the start of the dense reconstruction phase via OpenMVS.

github.com/OpenDroneMap/ODM

Support of 8bit multi band JPEG + 16bit single band TIFF images

opened 09:30PM - 04 Mar 21 UTC

zgyivi

enhancement help wanted

### How did you install OpenDroneMap? (Docker, natively, ...)? Docker as part… of WebODM ### What's your browser and operating system? (Copy/paste the output of https://www.whatismybrowser.com/) Firefox 86 on Ubuntu Linux ### What is the problem? DJI Multispectral camera provides 8bit RGB JPG files with EXIF data + 5 separate 16bit single band TIFF images When I try to process such a set OpenMVS/DensifyPointCloud reports error, because the undistorted RGB images are converted to 16bit 3band TIFF format and it cannot process that. ### What should be the expected behavior? If this is a feature request, please describe in detail the changes you think should be made to the code, citing files and lines where changes should be made, if possible. Undistorting should keep 8bit format for multiband images even if there are 16bit single band files in the set. ### How can we reproduce this? (What steps did you do to trigger the problem? What parameters are you using for processing? If possible please include a copy of your dataset uploaded on Google Drive or Dropbox. Be detailed) The problem occur on the image set which was shared by mapninja on community page [DJI Phantom 4 Pro Multipectral w/o RTK](https://community.opendronemap.org/t/dji-phantom-4-pro-multipectral-w-o-rtk/4978?u=zgyivi) Only the JPGs can be processed properly, but when all images added processing fails as described above. Consol log shows ([link to whole log](https://drive.google.com/file/d/1RdW8SGt7cC2HfRbV_bH9DV8ZPDp9ECpY/view?usp=sharing)): [INFO] Initializing ODM - Sat Feb 27 23:14:17 2021 [INFO] ============== [INFO] build_overviews: False [INFO] camera_lens: auto [INFO] cameras: {} [INFO] crop: 3 [INFO] debug: False [INFO] dem_decimation: 1 [INFO] dem_euclidean_map: False [INFO] dem_gapfill_steps: 3 [INFO] dem_resolution: 5 [INFO] depthmap_resolution: 640 [INFO] dsm: False [INFO] dtm: False [INFO] end_with: odm_report [INFO] fast_orthophoto: False [INFO] feature_quality: high [INFO] feature_type: sift [INFO] force_gps: False [INFO] gcp: None [INFO] geo: None [INFO] gps_accuracy: 10 [INFO] ignore_gsd: False [INFO] matcher_distance: 0 [INFO] matcher_neighbors: 8 [INFO] matcher_type: flann [INFO] max_concurrency: 16 [INFO] merge: all [INFO] mesh_octree_depth: 11 [INFO] mesh_size: 200000 [INFO] min_num_features: 8000 [INFO] name: 9357efe3-2a90-4fc2-b82c-be94813d33fb [INFO] optimize_disk_space: False [INFO] orthophoto_compression: DEFLATE [INFO] orthophoto_cutline: False [INFO] orthophoto_no_tiled: False [INFO] orthophoto_png: False [INFO] orthophoto_resolution: 5 [INFO] pc_classify: False [INFO] pc_csv: False [INFO] pc_ept: False [INFO] pc_filter: 2.5 [INFO] pc_las: False [INFO] pc_quality: medium [INFO] pc_rectify: False [INFO] pc_sample: 0 [INFO] primary_band: auto [INFO] project_path: /var/www/data [INFO] radiometric_calibration: camera [INFO] rerun: None [INFO] rerun_all: False [INFO] rerun_from: None [INFO] resize_to: 2048 [INFO] skip_3dmodel: False [INFO] skip_band_alignment: False [INFO] skip_report: False [INFO] sm_cluster: None [INFO] smrf_scalar: 1.25 [INFO] smrf_slope: 0.15 [INFO] smrf_threshold: 0.5 [INFO] smrf_window: 18.0 [INFO] split: 999999 [INFO] split_image_groups: None [INFO] split_overlap: 150 [INFO] texturing_data_term: gmi [INFO] texturing_outlier_removal_type: gauss_clamping [INFO] texturing_skip_global_seam_leveling: True [INFO] texturing_skip_local_seam_leveling: False [INFO] texturing_tone_mapping: none [INFO] tiles: False [INFO] time: False [INFO] use_3dmesh: False [INFO] use_exif: False [INFO] use_fixed_camera_params: False [INFO] use_hybrid_bundle_adjustment: False [INFO] verbose: False [INFO] ============== [INFO] Running dataset stage [INFO] Loading dataset from: /var/www/data/9357efe3-2a90-4fc2-b82c-be94813d33fb/images [INFO] Loading 486 images [INFO] Found 486 usable images [INFO] Parsing SRS header: WGS84 UTM 10N [INFO] Finished dataset stage [INFO] Running split stage [INFO] Normal dataset, will process all at once. [INFO] Finished split stage [INFO] Running merge stage [INFO] Normal dataset, nothing to merge. [INFO] Finished merge stage [INFO] Running opensfm stage [INFO] Reconstruction will use 81 images from auto band [INFO] Writing exif overrides [INFO] Maximum photo dimensions: 1600px [INFO] Altitude data detected, enabling it for GPS alignment [INFO] ['use_exif_size: no', 'flann_algorithm: KDTREE', 'feature_process_size: 800', 'feature_min_frames: 8000', 'processes: 16', 'matching_gps_neighbors: 8', 'matching_gps_distance: 0', 'optimize_camera_parameters: yes', 'undistorted_image_format: tif', 'bundle_outlier_filtering_type: AUTO', 'align_orientation_prior: vertical', 'triangulation_type: ROBUST', 'retriangulation_ratio: 2', 'feature_type: SIFT', 'use_altitude_tag: yes', 'align_method: auto', 'local_bundle_radius: 0'] [INFO] running /code/SuperBuild/src/opensfm/bin/opensfm extract_metadata "/var/www/data/9357efe3-2a90-4fc2-b82c-be94813d33fb/opensfm" 2021-02-27 23:14:23,295 INFO: Extracting EXIF for DJI_0030.JPG 2021-02-27 23:14:23,327 INFO: Extracting EXIF for DJI_0020.JPG 2021-02-27 23:14:23,348 INFO: Extracting EXIF for DJI_0040.JPG 2021-02-27 23:14:23,369 INFO: Extracting EXIF for DJI_0050.JPG ... [INFO] Undistorting /var/www/data/9357efe3-2a90-4fc2-b82c-be94813d33fb/opensfm ... 2021-02-27 23:21:06,859 DEBUG: Undistorting image DJI_0155.TIF 2021-02-27 23:21:06,861 DEBUG: Undistorting image DJI_0602.TIF 2021-02-27 23:21:06,865 DEBUG: Undistorting image DJI_0455.TIF 2021-02-27 23:21:06,867 DEBUG: Undistorting image DJI_0453.TIF 2021-02-27 23:21:06,868 DEBUG: Undistorting image DJI_0451.TIF 2021-02-27 23:21:06,872 DEBUG: Undistorting image DJI_0054.TIF 2021-02-27 23:21:06,876 DEBUG: Undistorting image DJI_0053.TIF 2021-02-27 23:21:06,881 DEBUG: Undistorting image DJI_0051.TIF 2021-02-27 23:21:06,887 DEBUG: Undistorting image DJI_0144.TIF 2021-02-27 23:21:06,895 DEBUG: Undistorting image DJI_0143.TIF ... 2021-02-27 23:23:01,227 DEBUG: Undistorting image DJI_0373.TIF 2021-02-27 23:23:01,378 DEBUG: Undistorting image DJI_0040.JPG 2021-02-27 23:23:01,534 DEBUG: Undistorting image DJI_0291.TIF 2021-02-27 23:23:01,539 DEBUG: Undistorting image DJI_0294.TIF 2021-02-27 23:23:02,542 DEBUG: Undistorting image DJI_0702.TIF 2021-02-27 23:23:02,901 DEBUG: Undistorting image DJI_0703.TIF 2021-02-27 23:23:03,235 DEBUG: Undistorting image DJI_0705.TIF [INFO] Restored reconstruction.json [INFO] Undistorting /var/www/data/9357efe3-2a90-4fc2-b82c-be94813d33fb/opensfm ... 2021-02-27 23:23:09,796 DEBUG: Undistorting the reconstruction 2021-02-27 23:23:15,243 DEBUG: Undistorting image DJI_0150.JPG 2021-02-27 23:23:15,243 DEBUG: Undistorting image DJI_0600.JPG 2021-02-27 23:23:15,243 DEBUG: Undistorting image DJI_0450.JPG 2021-02-27 23:23:15,243 DEBUG: Undistorting image DJI_0050.JPG 2021-02-27 23:23:15,243 DEBUG: Undistorting image DJI_0140.JPG 2021-02-27 23:23:15,244 DEBUG: Undistorting image DJI_0430.JPG 2021-02-27 23:23:15,244 DEBUG: Undistorting image DJI_0740.JPG ... 2021-02-27 23:23:40,559 DEBUG: Undistorting image DJI_0220.JPG 2021-02-27 23:23:40,559 DEBUG: Undistorting image DJI_0810.JPG 2021-02-27 23:23:40,896 DEBUG: Undistorting image DJI_0780.JPG 2021-02-27 23:23:41,218 DEBUG: Undistorting image DJI_0650.JPG [INFO] running /code/SuperBuild/src/opensfm/bin/opensfm export_visualsfm --points "/var/www/data/9357efe3-2a90-4fc2-b82c-be94813d33fb/opensfm" [INFO] Multiple bands found [INFO] Finished opensfm stage [INFO] Running openmvs stage [INFO] running /code/SuperBuild/src/opensfm/bin/opensfm export_openmvs "/var/www/data/9357efe3-2a90-4fc2-b82c-be94813d33fb/opensfm" [INFO] Running dense reconstruction. This might take a while. [INFO] running /code/SuperBuild/install/bin/OpenMVS/DensifyPointCloud "/var/www/data/9357efe3-2a90-4fc2-b82c-be94813d33fb/opensfm/undistorted/openmvs/scene.mvs" --resolution-level 2 --min-resolution 320 --max-resolution 1397 --max-threads 16 --number-views-fuse 2 -w "/var/www/data/9357efe3-2a90-4fc2-b82c-be94813d33fb/opensfm/undistorted/openmvs/depthmaps" -v 0 23:24:28 [App ] Build date: Feb 27 2021, 22:53:47 23:24:28 [App ] CPU: Intel(R) Xeon(R) CPU E5-2665 0 @ 2.40GHz (16 cores) 23:24:28 [App ] RAM: 15.62GB Physical Memory 0B Virtual Memory 23:24:28 [App ] OS: Linux 4.15.0-134-generic (x86_64) 23:24:28 [App ] SSE & AVX compatible CPU & OS detected 23:24:28 [App ] Command line: /var/www/data/9357efe3-2a90-4fc2-b82c-be94813d33fb/opensfm/undistorted/openmvs/scene.mvs --resolution-level 2 --min-resolution 320 --max-resolution 1397 --max-threads 16 --number-views-fuse 2 -w /var/www/data/9357efe3-2a90-4fc2-b82c-be94813d33fb/opensfm/undistorted/openmvs/depthmaps -v 0 23:24:28 [App ] error: failed loading image header 23:24:28 [App ] error: failed reloading image '/var/www/data/9357efe3-2a90-4fc2-b82c-be94813d33fb/opensfm/undistorted/images/DJI_0280.JPG.tif' 23:24:28 [App ] error: invalid project Traceback (most recent call last): File "/code/run.py", line 68, in <module> app.execute() File "/code/stages/odm_app.py", line 81, in execute self.first_stage.run() File "/code/opendm/types.py", line 338, in run self.next_stage.run(outputs) File "/code/opendm/types.py", line 338, in run self.next_stage.run(outputs) File "/code/opendm/types.py", line 338, in run self.next_stage.run(outputs) [Previous line repeated 1 more time] File "/code/opendm/types.py", line 319, in run self.process(self.args, outputs) File "/code/stages/openmvs.py", line 60, in process system.run('%s "%s" %s' % (context.omvs_densify_path, File "/code/opendm/system.py", line 79, in run raise Exception("Child returned {}".format(retcode)) Exception: Child returned 1

As far as I can tell, currently MicaSense’ imageprocessing library cannot be used to preprocess the imagery.

system · 7 May 2021 01:32

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.