Unknown error occured

Baldan_Sukhbaatar · 4 August 2021 00:21

Dear all

I am having environment with AWS and Lightning. In some dataset unknow error occurs (the dataset is valid drone images)

Here is short log:

Running global seam leveling:
Create matrices for optimization… Floating point exception (core dumped)

===== Dumping Info for Geeks (developers need this to fix bugs) =====
Child returned 136
Traceback (most recent call last):
File “/code/stages/odm_app.py”, line 89, in execute
self.first_stage.run()
File “/code/opendm/types.py”, line 340, in run
self.next_stage.run(outputs)
File “/code/opendm/types.py”, line 340, in run
self.next_stage.run(outputs)
File “/code/opendm/types.py”, line 340, in run
self.next_stage.run(outputs)
[Previous line repeated 3 more times]
File “/code/opendm/types.py”, line 321, in run
self.process(self.args, outputs)
File “/code/stages/mvstex.py”, line 108, in process
system.run(’"{bin}" “{nvm_file}” “{model}” “{out_dir}” ’
File “/code/opendm/system.py”, line 106, in run
raise SubprocessException(“Child returned {}”.format(retcode), retcode)
opendm.system.SubprocessException: Child returned 136

===== Done, human-readable information to follow… =====

[ERROR] The program exited with a strange error code. Please report it at https://community.opendronemap.org
90…100 - done.

Here is detailed log:

… output truncated at undefined lines …
2021-08-03 08:33:38,599 DEBUG: Setting pcolormesh
2021-08-03 08:33:41,926 DEBUG: locator: <matplotlib.colorbar._ColorbarAutoLocator object at 0x7fb19524ee20>
2021-08-03 08:33:41,926 DEBUG: Using auto colorbar locator <matplotlib.colorbar._ColorbarAutoLocator object at 0x7fb19524ee20> on colorbar
2021-08-03 08:33:41,926 DEBUG: Setting pcolormesh
[INFO] running /code/SuperBuild/install/bin/opensfm/bin/opensfm export_geocoords --reconstruction --proj “+proj=utm +zone=54 +datum=WGS84 +units=m +no_defs +type=crs” --offset-x 357563.0 --offset-y 4315175.0 “/var/www/data/165215df-be75-45ab-b40d-75d9aabe2ac3/opensfm”
[INFO] Updating /var/www/data/165215df-be75-45ab-b40d-75d9aabe2ac3/opensfm/config.yaml
[INFO] undistorted_image_max_size: 5472
[INFO] Undistorting /var/www/data/165215df-be75-45ab-b40d-75d9aabe2ac3/opensfm …
2021-08-03 08:35:04,509 DEBUG: Undistorting image DJI_0395.JPG
…
2021-08-03 08:41:24,453 DEBUG: Undistorting image DJI_0246.JPG
[INFO] running /code/SuperBuild/install/bin/opensfm/bin/opensfm export_visualsfm --points “/var/www/data/165215df-be75-45ab-b40d-75d9aabe2ac3/opensfm”
[INFO] running /code/SuperBuild/install/bin/opensfm/bin/opensfm export_ply --no-cameras --point-num-views “/var/www/data/165215df-be75-45ab-b40d-75d9aabe2ac3/opensfm”
[INFO] Finished opensfm stage
[INFO] Running odm_filterpoints stage
[INFO] Sampling points around a 0.01m radius
[INFO] Filtering /var/www/data/165215df-be75-45ab-b40d-75d9aabe2ac3/opensfm/reconstruction.ply (statistical, meanK 16, standard deviation 2.5)
[INFO] running pdal translate -i “/var/www/data/165215df-be75-45ab-b40d-75d9aabe2ac3/opensfm/reconstruction.ply” -o “/var/www/data/165215df-be75-45ab-b40d-75d9aabe2ac3/odm_filterpoints/point_cloud.ply” sample outlier range --writers.ply.sized_types=false --writers.ply.storage_mode=“little endian” --writers.ply.dims=“x=float,y=float,z=float,red=uchar,blue=uchar,green=uchar,views=uchar” --filters.sample.radius=0.01 --filters.outlier.method=“statistical” --filters.outlier.mean_k=16 --filters.outlier.multiplier=2.5 --filters.range.limits=“Classification![7:7]”
[INFO] Finished odm_filterpoints stage
[INFO] Running odm_meshing stage
[INFO] Writing ODM 2.5D Mesh file in: /var/www/data/165215df-be75-45ab-b40d-75d9aabe2ac3/odm_meshing/odm_25dmesh.ply
[WARNING] Maximum resolution set to GSD - 10.0% (22.3 cm / pixel, requested resolution was 2.0 cm / pixel)
[INFO] ODM 2.5D DSM resolution: 5.524142811583081
[INFO] Created temporary directory: /var/www/data/165215df-be75-45ab-b40d-75d9aabe2ac3/odm_meshing/tmp
[INFO] Creating DSM for 2.5D mesh
[INFO] running pdal info “/var/www/data/165215df-be75-45ab-b40d-75d9aabe2ac3/odm_filterpoints/point_cloud.ply” > “/tmp/tmppveuhyxp.json”
[INFO] Point cloud bounds are [minx: 88537.03125, maxx: 88603.58594] [miny: -302886.4062, maxy: -302743.5312]
[WARNING] Really low resolution DEM requested (13, 26) will set floor at 64 pixels. Resolution changed to 2.3937952183526687. The scale of this reconstruction might be off.
[INFO] DEM resolution is (64, 30), max tile size is 4096, will split DEM generation into 1 tiles
[INFO] Generating /var/www/data/165215df-be75-45ab-b40d-75d9aabe2ac3/odm_meshing/tmp/mesh_dsm_r6.770675326676417_x0_y0.tif (max, radius: 6.770675326676417, resolution: 2.3937952183526687)
[INFO] running pdal pipeline -i /tmp/tmpo_hoyuqs.json > /dev/null 2>&1
[INFO] running gdalbuildvrt -input_file_list “/var/www/data/165215df-be75-45ab-b40d-75d9aabe2ac3/odm_meshing/tmp/tiles_list.txt” “/var/www/data/165215df-be75-45ab-b40d-75d9aabe2ac3/odm_meshing/tmp/tiles.vrt”
0…10…20…30…40…50…60…70…80…90…100 - done.
[INFO] running gdal_translate -co NUM_THREADS=8 -co BIGTIFF=IF_SAFER --config GDAL_CACHEMAX 46.85% “/var/www/data/165215df-be75-45ab-b40d-75d9aabe2ac3/odm_meshing/tmp/tiles.vrt” “/var/www/data/165215df-be75-45ab-b40d-75d9aabe2ac3/odm_meshing/tmp/tiles.tmp.tif”
Input file size is 28, 60
0…10…20…30…40…50…60…70…80…90…100 - done.
[INFO] running gdal_translate -co NUM_THREADS=8 -co BIGTIFF=IF_SAFER --config GDAL_CACHEMAX 46.85% -outsize 10% 0 “/var/www/data/165215df-be75-45ab-b40d-75d9aabe2ac3/odm_meshing/tmp/tiles.tmp.tif” “/var/www/data/165215df-be75-45ab-b40d-75d9aabe2ac3/odm_meshing/tmp/tiles.small.tif”
0
…10…20…30…40…50
…60…70…80…
Input file size is 28, 60
0…10…20…30…40…50…60…70…80…90…100 - done.
[INFO] running gdalbuildvrt -resolution highest -r bilinear “/var/www/data/165215df-be75-45ab-b40d-75d9aabe2ac3/odm_meshing/tmp/merged.vrt” “/var/www/data/165215df-be75-45ab-b40d-75d9aabe2ac3/odm_meshing/tmp/tiles.small_filled.tif” “/var/www/data/165215df-be75-45ab-b40d-75d9aabe2ac3/odm_meshing/tmp/tiles.tmp.tif”
0…10…20…30…40…50…60…70…80…90…100 - done.
[INFO] running gdal_translate -co NUM_THREADS=8 -co TILED=YES -co BIGTIFF=IF_SAFER -co COMPRESS=DEFLATE --config GDAL_CACHEMAX 46.85% “/var/www/data/165215df-be75-45ab-b40d-75d9aabe2ac3/odm_meshing/tmp/merged.vrt” “/var/www/data/165215df-be75-45ab-b40d-75d9aabe2ac3/odm_meshing/tmp/tiles.tif”
Input file size is 28, 60
0…10…20…30…40…50…60…70…80…90…100 - done.
[INFO] Completed mesh_dsm.tif in 0:00:01.171814
[INFO] Sampling points from DSM: /var/www/data/165215df-be75-45ab-b40d-75d9aabe2ac3/odm_meshing/tmp/mesh_dsm.tif
[INFO] running “/code/SuperBuild/install/bin/dem2points” -inputFile “/var/www/data/165215df-be75-45ab-b40d-75d9aabe2ac3/odm_meshing/tmp/mesh_dsm.tif” -outputFile “/var/www/data/165215df-be75-45ab-b40d-75d9aabe2ac3/odm_meshing/tmp/dsm_points.ply” -skirtHeightThreshold 1.5 -skirtIncrements 0.2 -skirtHeightCap 100
[INFO] running “/code/SuperBuild/install/bin/PoissonRecon” --in “/var/www/data/165215df-be75-45ab-b40d-75d9aabe2ac3/odm_meshing/tmp/dsm_points.ply” --out “/var/www/data/165215df-be75-45ab-b40d-75d9aabe2ac3/odm_meshing/odm_25dmesh.dirty.ply” --depth 11 --pointWeight 4 --samplesPerNode 1.0 --threads 7 --maxMemory 47 --bType 2 --linearFit
[INFO] running “/code/SuperBuild/install/bin/OpenMVS/ReconstructMesh” -i “/var/www/data/165215df-be75-45ab-b40d-75d9aabe2ac3/odm_meshing/odm_25dmesh.dirty.ply” -o “/var/www/data/165215df-be75-45ab-b40d-75d9aabe2ac3/odm_meshing/odm_25dmesh.ply” --remove-spikes 0 --remove-spurious 20 --smooth 0 --target-face-num 400000
08:42:41 [App ] Build date: Jun 22 2021, 17:22:42
08:42:41 [App ] CPU: Intel(R) Xeon(R) CPU E3-1275 v5 @ 3.60GHz (8 cores)
08:42:41 [App ] RAM: 62.70GB Physical Memory 64.00GB Virtual Memory
08:42:41 [App ] OS: Linux 4.15.0-88-generic (x86_64)
08:42:41 [App ] SSE & AVX compatible CPU & OS detected
08:42:41 [App ] Command line: -i /var/www/data/165215df-be75-45ab-b40d-75d9aabe2ac3/odm_meshing/odm_25dmesh.dirty.ply -o /var/www/data/165215df-be75-45ab-b40d-75d9aabe2ac3/odm_meshing/odm_25dmesh.ply --remove-spikes 0 --remove-spurious 20 --smooth 0 --target-face-num 400000
08:42:41 [App ] Mesh loaded: 6268 vertices, 12484 faces (2ms)
08:42:41 [App ] Cleaned mesh: 6242 vertices, 11670 faces (22ms)
08:42:41 [App ] Cleaned mesh: 5834 vertices, 11667 faces (15ms)
08:42:41 [App ] Cleaned mesh: 5834 vertices, 11667 faces (9ms)
08:42:41 [App ] Scene saved (5ms):
0 images (32766 calibrated)
0 points, 5834 vertices, 11667 faces
08:42:41 [App ] Mesh saved: 5834 vertices, 11667 faces (2ms)
08:42:41 [App ] MEMORYINFO: {
08:42:41 [App ] VmPeak: 70952 kB
08:42:41 [App ] VmSize: 68508 kB
08:42:41 [App ] } ENDINFO
[INFO] Finished odm_meshing stage
[INFO] Running mvs_texturing stage
[INFO] Writing MVS Textured file in: /var/www/data/165215df-be75-45ab-b40d-75d9aabe2ac3/odm_texturing_25d/odm_textured_model_geo.obj
[INFO] running “/code/SuperBuild/install/bin/texrecon” “/var/www/data/165215df-be75-45ab-b40d-75d9aabe2ac3/opensfm/undistorted/reconstruction.nvm” “/var/www/data/165215df-be75-45ab-b40d-75d9aabe2ac3/odm_meshing/odm_25dmesh.ply” “/var/www/data/165215df-be75-45ab-b40d-75d9aabe2ac3/odm_texturing_25d/odm_textured_model_geo” -d gmi -o gauss_clamping -t none --no_intermediate_results --nadir_mode
/code/SuperBuild/install/bin/texrecon (built on Jun 22 2021, 17:23:19)
Load and prepare mesh:
Reading PLY: 5834 verts… 11667 faces… done.
Generating texture views:
NVM: Loading file…
NVM: Number of views: 363
NVM: Number of features: 216316

Loading 100%… done. (Took 33.229s)
Building adjacency graph:

Adding edges 100%… done. (Took 0.021s)
17499 total edges.
View selection:
Building BVH from 11667 faces… done. (Took: 12 ms)

Calculating face qualities 100%… done. (Took 33.659s)

Postprocessing face infos 100%… done. (Took 0s)
Maximum quality of a face within an image: 0
Clamping qualities to 0 within normalization.
Optimizing:
Time[s] Energy
0 11667
0 11667
0 11667
0 11667
0 11667
0 11667
0 11667
11667 faces have not been seen
Took: 33.733s
Generating texture patches:
Running… done. (Took 30.386s)
0 texture patches.
Running global seam leveling:
Create matrices for optimization… Floating point exception (core dumped)

===== Dumping Info for Geeks (developers need this to fix bugs) =====
Child returned 136
Traceback (most recent call last):
File “/code/stages/odm_app.py”, line 89, in execute
self.first_stage.run()
File “/code/opendm/types.py”, line 340, in run
self.next_stage.run(outputs)
File “/code/opendm/types.py”, line 340, in run
self.next_stage.run(outputs)
File “/code/opendm/types.py”, line 340, in run
self.next_stage.run(outputs)
[Previous line repeated 3 more times]
File “/code/opendm/types.py”, line 321, in run
self.process(self.args, outputs)
File “/code/stages/mvstex.py”, line 108, in process
system.run(’"{bin}" “{nvm_file}” “{model}” “{out_dir}” ’
File “/code/opendm/system.py”, line 106, in run
raise SubprocessException(“Child returned {}”.format(retcode), retcode)
opendm.system.SubprocessException: Child returned 136

===== Done, human-readable information to follow… =====

[ERROR] The program exited with a strange error code. Please report it at https://community.opendronemap.org
90…100 - done.

Saijin_Naib · 4 August 2021 03:17

Are you able to share the data and your processing parameters with us?

Baldan_Sukhbaatar · 4 August 2021 04:54

Dear Sajin

Thank you for fast reply.

https://drive.google.com/drive/folders/1_FB7NCVgmy2AiKrqxwMuVQRU7qE-aQuF?usp=sharing

The dataset is huge, therefore here is some faulty (*by webodm) image set.
Parameters are:
min-num-features: 2000, fast-orthophoto: true, skip-3dmodel: true

*min-num-features: we tested with 2k to 8k

Saijin_Naib · 7 August 2021 15:39

Well this set of images I don’t think can stitch. You’ve got about 28 images with at most 20ft of separation between them, really close to the ground.

What is the rest of the dataset like?

Saijin_Naib · 10 August 2021 14:56

I take it back! This part stitched quite nicely.

Options: cog: true, crop: 0, debug: true, dem-gapfill-steps: 4, dem-resolution: 1, dsm: true, mesh-size: 300000, min-num-features: 12000, orthophoto-resolution: 1, pc-classify: true, pc-ept: true, pc-geometric: true, pc-quality: high, use-3dmesh: true, verbose: true

Baldan_Sukhbaatar · 12 August 2021 03:04

Wow. It is great. So what I needed to do was choosing the right parameter options to run?
Got it. I set min-num-feature to 12000, that solved all issues.

Thank you very much.
But I am just wondering what is the ratio between min-num-feature and overlapping.
Since the system will be used by non professionals I have to suggest some rules to define parameters.

Saijin_Naib · 12 August 2021 04:43

I’m not sure there is a relationship, honestly.

–min-num-features is more about forcing the feature detection algorithm to extract more possible tiepoints per image. The more there are, the more likely they are to match in other images.

It also vastly increases memory consumption and processing time if you push it too high.

system · 11 September 2021 04:44

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.