Masking findings and problems

I’m still trying to get my sculpture photos to process. I thought masking non-essential features might make it run faster. So I’ve been playing with masking, both with mask files and with mattes directly on the images.

I’m using Windows 10 Pro on a system with 64GB memory running native WebODM 1.9.14 Build 62. I’ve also tested with Docker/WebODM 1.9.14 with similar results.

I don’t have any conclusion as to speed yet, just some interesting observations.

My subject is a complex bronze sculpture with lots of distractions in the background. Sky, water, sand, walkways and buildings that all seem like they would be better eliminated for matching purposes and to simplify the resulting textured mesh. Reflections are also a problem. I shot close to 400 images, starting with an overview (circled around the sculpture), then got increasing detail with later images. The first 20 or so images (usually) process nicely, although leave a lot of holes. I did some masking directly on the images (not a mask file) using PhotoShop. I quickly and approximately brushed over the backgrounds. I have also done some testing with more precise masks, either done manually, or using PhotoShop’s tools to refine the masks. I increased the precision of the masks because parts of the textured mesh were showing coloring and distortion from the background (grass and shrubbery), hoping that might eliminate the bleed from the background.

Processing options are:
debug: true, mesh-octree-depth: 13, mesh-size: 1000000, min-num-features: 30000, pc-quality: ultra, resize-to: -1, verbose: true

The unmasked images seem to (almost) always process, but I have never completed more than about 100 images because of the time it takes to do the matching. The masked images, especially the ones with more precise masks, usually fail with this message:

It looks like there might be one of the following problems:
	Not enough images
	Not enough overlap between images
	Images might be too blurry (common with phone cameras)
	The min-num-features task option is set too low, try increasing it by 25%
You can read more about best practices for capturing good images here.

Sometimes the detect_features section of the log file contains this error message. When this occurs, the match_features task always fails. Cause and effect or coincidence?

C:\WebODM\resources\app\apps\ODM\SuperBuild\install\bin\opensfm\opensfm\features.py:233: RuntimeWarning: invalid value encountered in true_divide
desc = np.sqrt(desc.T / s).T

The match_features fails with

cv2.error: OpenCV(4.5.0) D:\a\ODM\ODM\SuperBuild\src\opencv\modules\flann\include\opencv2\flann\kdtree_index.h:470: error: (-215:Assertion failed) result.full() in function 'cvflann::KDTreeIndex<struct cvflann::L2<float> >::getNeighbors'

I can provide the full traceback and/or the log file if needed.

Questions:

  1. It appears to me that masking does not always improve processing, and may cause problems. Has anyone else encountered situations where masking makes things worse? Are there guidelines on what to and what not to mask?

  2. Some of my images are blurry in areas because of inadequate depth of field. Should I attempt to mask the blurry areas?

  3. The sculpture is quite reflective. Should I mask the reflective areas? Usually there is another photo of the same area taken from a different angle with the reflections more controlled.

Thank you!

1 Like

Can you just leave all the images with bad reflections out of the dataset, or would that leave areas with not enough coverage?

Have you tried using a boundary file instead of masking? Just draw a circle around the sculpture here: http://geojson.io/ to produce the boundary file.

1 Like

I’ll try leaving more of the images with reflections out. Leaving all out would probably reduce the coverage too much. (Since it’s a bronze sculpture outdoors, most images have some reflections!) I’ve already cut my ~400 images down to 232. I may also try to reshoot it at a different time of day. The images I’m working with were shot mid-morning on a moderately overcast day.

Isn’t a boundary file processed after the feature extraction and image matching is complete? I can try a boundary file and see what happens.

Thanks for the suggestions.

2 Likes

Yes, but the end result will be a more compact model with no random bits of background around it

2 Likes