Seems like error in Point Cloud Rectification

ODM
,
#1

After processing for almost 2 days, I got this error:

{
            "command": "pdal translate -i /var/www/data/157d1ed7-9c80-4b81-b692-062d1f8cedac/odm_georeferencing/odm_georeferenced_model.laz -o /var/www/data/157d1ed7-9c80-4b81-b692-062d1f8cedac/odm_georeferencing/tmp.las",
            "exitCode": 0
        }
    ],
    "success": false,
    "error": {
        "code": 1,
        "message": "Error rectifying ground in file /var/www/data/157d1ed7-9c80-4b81-b692-062d1f8cedac/odm_georeferencing/odm_georeferenced_model.laz: QH6235 qhull error (qh_memalloc): negative request size (-778240640).  Did int overflow due to high-D?\n\nWhile executing:  | qhull i Qt\nOptions selected for Qhull 2019.1.r 2019/06/21:\n  run-id 1403530678  incidence  Qtriangulate  _pre-merge  _zero-centrum\n  _max-width 3.7e+03  Error-roundoff 2.2e-09  _one-merge 1.1e-08\n  _near-inside 5.4e-08  Visible-distance 4.3e-09  U-max-coplanar 4.3e-09\n  Width-outside 8.6e-09  _wide-facet 2.6e-08  _maxoutside 1.3e-08\n"
    },
    "stackTrace": [
        "Traceback (most recent call last):",
        "File \"/code/opendm/dem/commands.py\", line 67, in rectify",
        "run_rectification(",
        "File \"/code/opendm/dem/ground_rectification/rectify.py\", line 24, in run_rectification",
        "point_cloud = extend_cloud(point_cloud, kwargs['extend_plan'], kwargs['extend_grid_distance'], kwargs['min_points'], kwargs['min_area'])",
        "File \"/code/opendm/dem/ground_rectification/rectify.py\", line 61, in extend_cloud",
        "bounds = calculate_convex_hull_bounds(ground_cloud.get_xy())",
        "File \"/code/opendm/dem/ground_rectification/bounds/utils.py\", line 6, in calculate_convex_hull_bounds",
        "hull = ConvexHull(points)",
        "File \"qhull.pyx\", line 2431, in scipy.spatial.qhull.ConvexHull.__init__",
        "File \"qhull.pyx\", line 356, in scipy.spatial.qhull._Qhull.__init__",
        "scipy.spatial.qhull.QhullError: QH6235 qhull error (qh_memalloc): negative request size (-778240640).  Did int overflow due to high-D?",
        "",
        "While executing:  | qhull i Qt",
        "Options selected for Qhull 2019.1.r 2019/06/21:",
        "run-id 1403530678  incidence  Qtriangulate  _pre-merge  _zero-centrum",
        "_max-width 3.7e+03  Error-roundoff 2.2e-09  _one-merge 1.1e-08",
        "_near-inside 5.4e-08  Visible-distance 4.3e-09  U-max-coplanar 4.3e-09",
        "Width-outside 8.6e-09  _wide-facet 2.6e-08  _maxoutside 1.3e-08",
        "",
        "",
        "During handling of the above exception, another exception occurred:",
        "",
        "Traceback (most recent call last):",
        "File \"/code/stages/odm_app.py\", line 88, in execute",
        "self.first_stage.run()",
        "File \"/code/opendm/types.py\", line 378, in run",
        "self.next_stage.run(outputs)",
        "File \"/code/opendm/types.py\", line 378, in run",
        "self.next_stage.run(outputs)",
        "File \"/code/opendm/types.py\", line 378, in run",
        "self.next_stage.run(outputs)",
        "[Previous line repeated 6 more times]",
        "File \"/code/opendm/types.py\", line 359, in run",
        "self.process(self.args, outputs)",
        "File \"/code/stages/odm_dem.py\", line 76, in process",
        "commands.rectify(dem_input, args.debug)",
        "File \"/code/opendm/dem/commands.py\", line 84, in rectify",
        "raise Exception(\"Error rectifying ground in file %s: %s\" % (lasFile, str(e)))",
        "Exception: Error rectifying ground in file /var/www/data/157d1ed7-9c80-4b81-b692-062d1f8cedac/odm_georeferencing/odm_georeferenced_model.laz: QH6235 qhull error (qh_memalloc): negative request size (-778240640).  Did int overflow due to high-D?",
        "",
        "While executing:  | qhull i Qt",
        "Options selected for Qhull 2019.1.r 2019/06/21:",
        "run-id 1403530678  incidence  Qtriangulate  _pre-merge  _zero-centrum",
        "_max-width 3.7e+03  Error-roundoff 2.2e-09  _one-merge 1.1e-08",
        "_near-inside 5.4e-08  Visible-distance 4.3e-09  U-max-coplanar 4.3e-09",
        "Width-outside 8.6e-09  _wide-facet 2.6e-08  _maxoutside 1.3e-08",
        "",
        ""
    ],

Is there a problem with pc-rectify flag?

I was just checking the log and found "odmVersion": "2.8.8".

I run the following command to start ODM with GPU:
sudo docker run -p 3000:3000 --gpus all opendronemap/nodeodm:gpu

What command to run to get the latest odm version?

1 Like
#2

Sorry. I figured it out myself only.

1 Like
#3

I have updated odm and re-running the dataset. :crossed_fingers:

2 Likes
#4

what was the command to update? or did you delete the docker image and rerun?

2 Likes
#5

yes

1 Like
#6

I have updated the odm version to 3.0.2. But I am getting the same error only.

{
            "command": "\"/code/SuperBuild/install/bin/texrecon\" \"/var/www/data/b0807801-8640-41bd-bee7-4e22b3a6a87c/opensfm/undistorted/reconstruction.nvm\" \"/var/www/data/b0807801-8640-41bd-bee7-4e22b3a6a87c/odm_meshing/odm_25dmesh.ply\" \"/var/www/data/b0807801-8640-41bd-bee7-4e22b3a6a87c/odm_texturing_25d/odm_textured_model_geo\" -d gmi -o gauss_clamping -t none --no_intermediate_results    --nadir_mode  --max_texture_size=8192 ",
            "exitCode": 0,
            "output": [
                "",
                "Blending texture patches 100%... done. (Took 87.89s)",
                "Generating texture atlases:",
                "Sorting texture patches... done.",
                "",
                "Working on atlas 104 100%... done.",
                "Finalizing texture atlases... done. (Took: 0s)",
                "Building objmodel:",
                "Saving model... done.",
                "Whole texturing procedure took: 1657.39s"
            ]
        },
        {
            "command": "pdal translate -i \"/var/www/data/b0807801-8640-41bd-bee7-4e22b3a6a87c/odm_filterpoints/point_cloud.ply\" -o \"/var/www/data/b0807801-8640-41bd-bee7-4e22b3a6a87c/odm_georeferencing/odm_georeferenced_model.laz\" ferry transformation --filters.ferry.dimensions=\"views => UserData\" --writers.las.compression=\"lazip\" --filters.transformation.matrix=\"1 0 0 343683.0 0 1 0 2982837.0 0 0 1 0 0 0 0 1\" --writers.las.offset_x=343683.0 --writers.las.offset_y=2982837.0 --writers.las.scale_x=0.001 --writers.las.scale_y=0.001 --writers.las.scale_z=0.001 --writers.las.offset_z=0 --writers.las.a_srs=\"+proj=utm +zone=43 +datum=WGS84 +units=m +no_defs +type=crs\"",
            "exitCode": 0
        },
        {
            "command": "ogr2ogr -overwrite -f GPKG -a_srs \"+proj=utm +zone=43 +datum=WGS84 +units=m +no_defs +type=crs\" /var/www/data/b0807801-8640-41bd-bee7-4e22b3a6a87c/odm_georeferencing/odm_georeferenced_model.bounds.gpkg /var/www/data/b0807801-8640-41bd-bee7-4e22b3a6a87c/odm_georeferencing/odm_georeferenced_model.bounds.geojson",
            "exitCode": 0
        },
        {
            "command": "pdal translate -i \"/var/www/data/b0807801-8640-41bd-bee7-4e22b3a6a87c/odm_georeferencing/odm_georeferenced_model.laz\" -o \"/var/www/data/b0807801-8640-41bd-bee7-4e22b3a6a87c/odm_georeferencing/odm_georeferenced_model.las\" ",
            "exitCode": 0
        },
        {
            "command": "pdal translate -i /var/www/data/b0807801-8640-41bd-bee7-4e22b3a6a87c/odm_georeferencing/odm_georeferenced_model.laz -o /var/www/data/b0807801-8640-41bd-bee7-4e22b3a6a87c/odm_georeferencing/odm_georeferenced_model.laz smrf --filters.smrf.scalar=1.25 --filters.smrf.slope=0.15 --filters.smrf.threshold=0.5 --filters.smrf.window=18.0",
            "exitCode": 0
        }
    ],
    "success": false,
    "error": {
        "code": 1,
        "message": "Error rectifying ground in file /var/www/data/b0807801-8640-41bd-bee7-4e22b3a6a87c/odm_georeferencing/odm_georeferenced_model.laz: QH6235 qhull error (qh_memalloc): negative request size (-781368024).  Did int overflow due to high-D?\n\nWhile executing:  | qhull i Qt\nOptions selected for Qhull 2019.1.r 2019/06/21:\n  run-id 1581665235  incidence  Qtriangulate  _pre-merge  _zero-centrum\n  _max-width 3.7e+03  Error-roundoff 2.2e-09  _one-merge 1.1e-08\n  _near-inside 5.4e-08  Visible-distance 4.3e-09  U-max-coplanar 4.3e-09\n  Width-outside 8.6e-09  _wide-facet 2.6e-08  _maxoutside 1.3e-08\n"
    },
    "stackTrace": [
        "Traceback (most recent call last):",
        "File \"/code/opendm/dem/commands.py\", line 54, in rectify",
        "run_rectification(",
        "File \"/code/opendm/dem/ground_rectification/rectify.py\", line 24, in run_rectification",
        "point_cloud = extend_cloud(point_cloud, kwargs['extend_plan'], kwargs['extend_grid_distance'], kwargs['min_points'], kwargs['min_area'])",
        "File \"/code/opendm/dem/ground_rectification/rectify.py\", line 61, in extend_cloud",
        "bounds = calculate_convex_hull_bounds(ground_cloud.get_xy())",
        "File \"/code/opendm/dem/ground_rectification/bounds/utils.py\", line 6, in calculate_convex_hull_bounds",
        "hull = ConvexHull(points)",
        "File \"_qhull.pyx\", line 2436, in scipy.spatial._qhull.ConvexHull.__init__",
        "File \"_qhull.pyx\", line 357, in scipy.spatial._qhull._Qhull.__init__",
        "scipy.spatial._qhull.QhullError: QH6235 qhull error (qh_memalloc): negative request size (-781368024).  Did int overflow due to high-D?",
        "",
        "While executing:  | qhull i Qt",
        "Options selected for Qhull 2019.1.r 2019/06/21:",
        "run-id 1581665235  incidence  Qtriangulate  _pre-merge  _zero-centrum",
        "_max-width 3.7e+03  Error-roundoff 2.2e-09  _one-merge 1.1e-08",
        "_near-inside 5.4e-08  Visible-distance 4.3e-09  U-max-coplanar 4.3e-09",
        "Width-outside 8.6e-09  _wide-facet 2.6e-08  _maxoutside 1.3e-08",
        "",
        "",
        "During handling of the above exception, another exception occurred:",
        "",
        "Traceback (most recent call last):",
        "File \"/code/stages/odm_app.py\", line 81, in execute",
        "self.first_stage.run()",
        "File \"/code/opendm/types.py\", line 386, in run",
        "self.next_stage.run(outputs)",
        "File \"/code/opendm/types.py\", line 386, in run",
        "self.next_stage.run(outputs)",
        "File \"/code/opendm/types.py\", line 386, in run",
        "self.next_stage.run(outputs)",
        "[Previous line repeated 6 more times]",
        "File \"/code/opendm/types.py\", line 365, in run",
        "self.process(self.args, outputs)",
        "File \"/code/stages/odm_dem.py\", line 75, in process",
        "commands.rectify(dem_input, False)",
        "File \"/code/opendm/dem/commands.py\", line 61, in rectify",
        "raise Exception(\"Error rectifying ground in file %s: %s\" % (lasFile, str(e)))",
        "Exception: Error rectifying ground in file /var/www/data/b0807801-8640-41bd-bee7-4e22b3a6a87c/odm_georeferencing/odm_georeferenced_model.laz: QH6235 qhull error (qh_memalloc): negative request size (-781368024).  Did int overflow due to high-D?",
        "",
        "While executing:  | qhull i Qt",
        "Options selected for Qhull 2019.1.r 2019/06/21:",
        "run-id 1581665235  incidence  Qtriangulate  _pre-merge  _zero-centrum",
        "_max-width 3.7e+03  Error-roundoff 2.2e-09  _one-merge 1.1e-08",
        "_near-inside 5.4e-08  Visible-distance 4.3e-09  U-max-coplanar 4.3e-09",
        "Width-outside 8.6e-09  _wide-facet 2.6e-08  _maxoutside 1.3e-08",
        "",
        ""
    ],

If someone can suggest, what should I take the next steps?

These are parameters related to Point Cloud

        "pc_classify": true,
        "pc_copc": false,
        "pc_csv": false,
        "pc_ept": false,
        "pc_filter": 0.0,
        "pc_las": true,
        "pc_quality": "medium",
        "pc_rectify": true,
        "pc_sample": 0,
        "pc_tile": false,
1 Like
#7

How big is your Task in images and area?

#8

It is roughly around 4 sq. km. and 2.2k images

1 Like
#9

Iā€™m wondering if we donā€™t have trouble with --pc-rectify on larger Tasks/Task areasā€¦ Iā€™ve been trying to figure out if there is anything systematic in when it fails. Thatā€™s the only thing that seems consistent.

#10

I have removed pc-rectify parameters and reprocessing it. :crossed_fingers:

1 Like
#11

I can no longer process larger datasets on my spaceheater without split-merge. I havenā€™t had time to troubleshoot fully yet as it is a volunteer project and I have too many other pans on the fire at the moment, but I have little doubt weā€™ve broken things for larger datasets. I have a strong leaning toward adding a flag to turn it off as proposed in as a possible option in the original pull request.

2 Likes
#12

hmmmmmm,

Damn, Iā€™m 125 hours into a task I expect to take 300 hours (nearly 23000 images), and I have set: pc-rectify: true :sob:

3 Likes
#13

Oh boyā€¦ .

Well, letā€™s see if I wrote this correctly:

1 Like
#14

Probably not relevant, but all my previous failures due to PC rectify were with non-mapping tasks, ie creating 3D models, in an attempt to get up, down etc correctly orientated when there was no ground/plane surface imaged from above in the images, and no GPS data in EXIF.
Iā€™m not using split-merge in the current task, but it is sounding like I should call it quits and let my CPU cool down (currently 50-70C) in this current spell of hot weather;)

1 Like
#15

Is pc-rectify causing the problem?

I have to process one large dataset of around 3.7k images and need a good DTM. Should I skip pc-rectify parameter?

1 Like
#16

No. Throw the new ā€“pc-skip-geometric flag and report back.

2 Likes