Forest orthos produced from same test images show different results


#1

I have four orthos produced from the same 119 test images flown with >80% overlap at tree canopy with same yaml.settings. Images were take Oct/16 mid-day, so sun was at lower angle creating strong shadows. 3 orthos were produced on same amd64 workstation with 32 vcores. J3455 was produced on 4 core Atom SBC

Each shows wildly differing results to the point of being unusable to observe tree health and location. Some trees simply disappear, others are quite tilted in one ortho straight in next, there are some mirages, trees do not align with ground observations even though natural ground points line up +/- 5 meters.

PNG ortho images will not upload to this posting

Any thoughts on how to improve the output and get some consistency would be welcome


#2

I don’t think you will be able to achieve 100% deterministic results across separate machines (or even across separate runs). I’m pretty sure certain calculations (especially in the texturing module) contain some elements of randomness.

On the plus side, you might be able to get better orthos once the work for https://github.com/OpenDroneMap/OpenDroneMap/projects/2 is completed.

For attaching PNGs try Google Drive or Dropbox.


#3

Thanks for input.

All but one ortho is produced on same workstation. Trees appear in one ortho, completely absent in another (0% deterministic?). Nothing is lining up with the field transects, allowing for GPS error. We have located & classified trees for health - green healthy, green attacked, red, gray, dead. Gray/red trees (easy markers) move all over the ortho by 20 - 30+ meters or even just disappear. In one ortho have a full tree crown right in a skid trail.

Tried 25d_mesh, but just produces random art images, don’t even look like trees. PMVS produces an ortho but has much lower detail/resolution and tree location is questionable, with trees appearing when don’t show in another view. Used both together, but still very distorted tree images.

Current settings are:
#resize_to: 2400
resize_to: 2000
#start_with: ‘resize’
#end_with: ‘odm_orthophoto’
#rerun_all: False
#zip_results: False
#verbose: False
verbose: True
#time: False
time: True
#min_num_features: 4000 #default
min_num_features: 15000
#matcher_threshold: 2.0 # zombie code, does nothing
#matcher_ratio: 0.6
#matcher_neighbors: 8
matcher_neighbors: 12
#matcher_distance: 0
matcher_distance: 250
#use_pmvs: False # The cmvs/pmvs settings only matter if ‘Enabled’ is set to True
#cmvs_maximages: 500
#pmvs_level: 1
#pmvs_csize: 2
#pmvs_threshold: 0.7
#pmvs_wsize: 7
#pmvs_min_images: 3
#pmvs_num_cores: 4 # by default this is set to $(nproc)
#mesh_size: 100000
mesh_size: 30000
#mesh_octree_depth: 9 #default - when pulls images off
mesh_octree_depth: 5
#mesh_samples: 1.0
#mesh_solver_divide: 9
mesh_remove_outliers: 3
#texturing_data_term: ‘gmi’
#texturing_outlier_removal_type: ‘gauss_clamping’
#texturing_skip_visibility_test: False
#texturing_skip_global_seam_leveling: False
#texturing_skip_local_seam_leveling: False
#texturing_skip_hole_filling: False
#texturing_keep_unseen_faces: False #default
#texturing_keep_unseen_faces: True #? adds shelf to mesh around textured section
#texturing_tone_mapping: ‘none’
#gcp: !!null # YAML tag for None
#use_exif: False # Set to True if you have a GCP file (it auto-detects) and want to use EXIF
use_exif: True
#dtm: False # Use this tag to build a DTM (Digital Terrain Model
#dsm: False # Use this tag to build a DSM (Digital Surface Model
#dem-gapfill-steps: 4
#dem-resolution: 0.1
#dem-maxangle:20
#dem-maxsd: 2.5
#dem-approximate: False
#dem-decimation: 1
dem-terrain-type: ComplexForest
#orthophoto_resolution: 20.0 # default Pixels/meter 20.0px/m = 5.0cm/px
orthophoto_resolution: 30.0
#orthophoto_target_srs: !!null # Currently does nothing
#orthophoto_no_tiled: False
orthophoto_no_tiled: True
#orthophoto_compression: DEFLATE # Options are [JPEG, LZW, PACKBITS, DEFLATE, LZMA, NONE] Don’t change unless you know what you are doing

Not sure what other parameters I should tweak to improve the forest orthos?

Will push test images and produced orthos to a shared drive and post.


#4

Neither the SfM portion nor the texturing are deterministic, but the aforementioned orthophoto improvements should improve results overall, hopefully resulting in some convergence.


#5

Mm, what happens if you change texturing_data_term from gmi to area? Do you get better results?


#6

Are you saying that there is no determinism in the formation of the produced ortho and point cloud? While I don’t expect 100%, how could an engineer or forester utilize ODM if results are non-deterministic?

Results just become pretty pictures if there is no repeatability. How can a forester have any confidence in their log yard volumes if the results are all over the map? A researcher in tree location and condition? A civil engineer in their tailings pile volume?

I can adjust for locational shifts in the whole tree stand, but not for random tree locations that do not align with plotted ground observations.

Is this a product of the software engineering? How do we tight this up? Or is it the nature of SfM?


#7

I am producing a fresh set of orthos with my settings to test the repeatability.

Tuned my test images to be all rotated north, which has yielded an ortho with seemingly better alignment to the ground observations. Will compare the next set of runs orthos.

Three runs, zoomed to a survey transect with plotted trees. GPS location to base of tree.
Red track is flight path
Cameras are where image was taken
Dotted purple line is the plotted transect
White line on the ground is actual transect line - ~3 meter error according to GPS
Green dots are live healthy trees
Light green dots are attacked but still alive trees
Red dots are red dead trees
White dots are gray dead trees
nodeodm-test-june1220181122-amd64

nodeodm-test-june1220181233-amd64

nodeodm-test-june1220181409-amd64

Notice the shift of trees, but more important is the complete appearance/disappearance of trees.

Here is the original image of roughly the centre of the above plot
2017_1016_133534_178


#8

Interesting, when build orthos from a 10 image subset that covers the transect plot area, objects, aka trees, in ortho are more consistently placed across the three generated images. Still some tree tilt from which image tree was pulled from and location shifting likely due to GPS error. ODM builds the orthos et al in ~5 min vs 53 min for 119 images.

Only the transect start and end were GPS, rest were all directly measured down/off transect line. So any GPS error should shift all points together. All trees inside the plot box where measured and classified, aka should have a corresponding dot if existed in ground survey.

nodeodm-test-june1220181623-amd64-transect-3

nodeodm-test-june1220181635-amd64-transect-3

nodeodm-test-june1220181649-amd64-transect-3

Still have trees appearing/disappearing between orthos, but less extreme.


#9

With 25d_mesh & pmvs -> still have distorted tree crowns showing as whirls, but not so abstract. Only effects a few of the more outlier trees compared to 25d_mesh use on whole set of images

nodeodm-test-june1220181741-amd64-transect-3-25d-pvms


#10

PMVS only on 10 image subset - yields the most stable orthos across 3 runs; still minor tree shift and a couple of questionable tree placements, noticeable missing tree in one ortho to the right; overall seeing a usable result once plots are realigned for GPS error

Processing time is also much quicker: 05:00 minutes vs 02:50, over 40% faster to produce full ODM run

nodeodm-test-june1220181759-amd64-transect-3-pmvs

nodeodm-test-june1220181810-amd64-transect-3-pmvs

nodeodm-test-june1220181823-amd64-transect-3-pmvs


#11

For completeness, here are three runs of the whole 119 image test set with pmvs:
nodeodm-test-june1220181849-amd64-pmvs

nodeodm-test-june1220181939-amd64-pmvs

nodeodm-test-june1220182026-amd64-pmvs

The small sub-set of input images that covers only the plot area seems to produce more repeatable output.

Note the almost double error between the white transect line and the plotted transect line building ortho with whole 119 images.


#12

My guess is that these inconsistencies are the result of not having a good mesh. Lower mesh values (5 is what you have these set to) are a useful trick to getting good orthos under certain circumstances, but in use cases like this are likely to cause a variety of texturing inconsistencies.

How tall are your trees, and what height did you fly at? (My hunch is the only solution is to wait for @pierotofy’s aforementioned fixes, but these details will help make that determination).


#13

I generally fly forest surveys at 120m AGL, trees are medium height ~100ft, overlap to crowns >80%

Mesh parameters used
#mesh_size: 100000 # default
mesh_size: 30000
#mesh_octree_depth: 9 #default - when pulls images off
mesh_octree_depth: 5
#mesh_samples: 1.0
#mesh_solver_divide: 9 # default
#mesh_solver_divide: 6
mesh_remove_outliers: 3

If memory serves, higher mesh_octree_depth values cause tearing and holes in the ortho. What is the effect of increasing or decreasing the mesh-size?


#14

Mvs texturing is not deterministic. https://github.com/nmoehrle/mvs-texturing/blob/413c4dce15c4a51052e25f65c6a0c6ca93f8208a/libs/tex/view_selection.cpp#L114

Could be interesting to toggle that flag to see if you get consistent results.


#15

Can do. Will build new node-odm with that flag change and test. Where does MVS fit into the stack? Should I run with or without PMVS?


#16

Here are three runs with MVS view flag = true on the 10 image sub-set. PMVS is not on, settings are as above.
nodeodm-test-june1320181605-amd64-mvs

nodeodm-test-june1320181617-amd64-mvs

nodeodm-test-june1320181629-amd64-mvs

PMVS of sub-set images is still the most repeatable. With MVS, still have trees appearing/disappearing, tree shifts in location. Also a bit of mirage is showing. Crisper detail, but other problems make this choice not viable. PMVS is also much quicker: 02:59 vs 05:13


#17

Here is MVS + PMVS. Not sure what difference it really makes, or if MVS & PMVS work together in the code. Time to run: 02:53 vs 05:13, so ~45% faster
nodeodm-test-june1320181653-amd64-mvs-pmvs


#18

And I think the work that Piero referenced above (https://github.com/OpenDroneMap/OpenDroneMap/projects/2) will make even more of a difference than PMVS, but glad to see some progress in advance of that.