Inspired by the results of ResDepth-stereo, I’ve decided to explore a perhaps simpler approach to DSM refinement that does not require machine learning.
The idea is that smoothing should improve the quality of elevation models for orthophoto rendering purposes, but smoothing should only be applied to sharp corner areas (e.g. edge of buildings), otherwise there’s loss of detail and other artifacts appear:
Stereo images can provide a strong signal to mark edges and there’s a variety of algorithms to extract edges/lines from images. In the attempted process, I extract edges from the RGB images and orthorectify them onto the target DSM. A redundancy counter is used to remove outliers (an edge needs to be seen by at least 2 different images to be included in the resulting mask).
Example with sobel:
Example with LSD (line segment detection):
The lines can then be used as a mask for smoothing operations on the original DSM, thus refining only edges and leaving other parts of the DSM untouched.
I will post results here as I go, but looks very promising.