PDAL: filters.range: Invalid dimension name in 'limits' option: 'Classification'

Saijin_Naib · 13 September 2021 13:37

Getting an interesting failure in PDAL the past few days across about 5 datasets when processing on Lightning against v.2.6.2.

The job will process fine until here and then just hang until 48ish hours have passed when I believe it gets auto-killed by Lightning.

What I think to be the salient line:

PDAL: filters.range: Invalid dimension name in 'limits' option: 'Classification'.

Processing Parameters:

Options: cog: true, crop: 0, debug: true, dem-gapfill-steps: 4, dem-resolution: 1, dsm: true, gps-accuracy: 5, matcher-neighbors: 16, mesh-size: 300000, min-num-features: 16000, orthophoto-resolution: 1, pc-classify: true, pc-ept: true, pc-filter: 0, pc-geometric: true, pc-quality: high, use-3dmesh: true, verbose: true

PDAL portion of Processing Log:

22:46:35 [App     ] MEMORYINFO: {
22:46:35 [App     ] 	VmPeak:	11442740 kB
22:46:35 [App     ] 	VmSize:	10548540 kB
22:46:35 [App     ] } ENDINFO
[INFO]    Finished openmvs stage
[INFO]    Running odm_filterpoints stage
[INFO]    Sampling points around a 0.01m radius
[INFO]    Splitting point cloud filtering in chunks of 250000 vertices
[INFO]    running pdal split -i "/var/www/data/d2e7a62a-9227-48f5-8e3f-1249d6588fb9/opensfm/undistorted/openmvs/scene_dense_dense_filtered.ply" -o "/var/www/data/d2e7a62a-9227-48f5-8e3f-1249d6588fb9/odm_filterpoints/parts/part.ply" --capacity 250000 --writers.ply.sized_types=false --writers.ply.storage_mode="little endian" --writers.ply.dims="x=float,y=float,z=float,nx=float,ny=float,nz=float,red=uchar,blue=uchar,green=uchar,views=uchar"
[INFO]    Sampling points around a 0.01m radius
[INFO]    running pdal translate -i "/var/www/data/d2e7a62a-9227-48f5-8e3f-1249d6588fb9/odm_filterpoints/parts/part_201.ply" -o "/var/www/data/d2e7a62a-9227-48f5-8e3f-1249d6588fb9/odm_filterpoints/parts/part_201_filtered.ply" sample range --writers.ply.sized_types=false --writers.ply.storage_mode="little endian" --writers.ply.dims="x=float,y=float,z=float,nx=float,ny=float,nz=float,red=uchar,blue=uchar,green=uchar,views=uchar" --filters.sample.radius=0.01 --filters.range.limits="Classification![7:7]"
[INFO]    Sampling points around a 0.01m radius
[INFO]    Sampling points around a 0.01m radius
[INFO]    Sampling points around a 0.01m radius
[INFO]    Sampling points around a 0.01m radius
[INFO]    Sampling points around a 0.01m radius
[INFO]    Sampling points around a 0.01m radius
[INFO]    Sampling points around a 0.01m radius
[INFO]    Sampling points around a 0.01m radius
[INFO]    Sampling points around a 0.01m radius
[INFO]    Sampling points around a 0.01m radius
[INFO]    Sampling points around a 0.01m radius
[INFO]    Sampling points around a 0.01m radius
[INFO]    Sampling points around a 0.01m radius
[INFO]    Sampling points around a 0.01m radius
[INFO]    running pdal translate -i "/var/www/data/d2e7a62a-9227-48f5-8e3f-1249d6588fb9/odm_filterpoints/parts/part_8.ply" -o "/var/www/data/d2e7a62a-9227-48f5-8e3f-1249d6588fb9/odm_filterpoints/parts/part_8_filtered.ply" sample range --writers.ply.sized_types=false --writers.ply.storage_mode="little endian" --writers.ply.dims="x=float,y=float,z=float,nx=float,ny=float,nz=float,red=uchar,blue=uchar,green=uchar,views=uchar" --filters.sample.radius=0.01 --filters.range.limits="Classification![7:7]"
[INFO]    running pdal translate -i "/var/www/data/d2e7a62a-9227-48f5-8e3f-1249d6588fb9/odm_filterpoints/parts/part_33.ply" -o "/var/www/data/d2e7a62a-9227-48f5-8e3f-1249d6588fb9/odm_filterpoints/parts/part_33_filtered.ply" sample range --writers.ply.sized_types=false --writers.ply.storage_mode="little endian" --writers.ply.dims="x=float,y=float,z=float,nx=float,ny=float,nz=float,red=uchar,blue=uchar,green=uchar,views=uchar" --filters.sample.radius=0.01 --filters.range.limits="Classification![7:7]"
[INFO]    Sampling points around a 0.01m radius
PDAL: filters.range: Invalid dimension name in 'limits' option: 'Classification'.

[INFO]    running pdal translate -i "/var/www/data/d2e7a62a-9227-48f5-8e3f-1249d6588fb9/odm_filterpoints/parts/part_202.ply" -o "/var/www/data/d2e7a62a-9227-48f5-8e3f-1249d6588fb9/odm_filterpoints/parts/part_202_filtered.ply" sample range --writers.ply.sized_types=false --writers.ply.storage_mode="little endian" --writers.ply.dims="x=float,y=float,z=float,nx=float,ny=float,nz=float,red=uchar,blue=uchar,green=uchar,views=uchar" --filters.sample.radius=0.01 --filters.range.limits="Classification![7:7]"
[INFO]    running pdal translate -i "/var/www/data/d2e7a62a-9227-48f5-8e3f-1249d6588fb9/odm_filterpoints/parts/part_164.ply" -o "/var/www/data/d2e7a62a-9227-48f5-8e3f-1249d6588fb9/odm_filterpoints/parts/part_164_filtered.ply" sample range --writers.ply.sized_types=false --writers.ply.storage_mode="little endian" --writers.ply.dims="x=float,y=float,z=float,nx=float,ny=float,nz=float,red=uchar,blue=uchar,green=uchar,views=uchar" --filters.sample.radius=0.01 --filters.range.limits="Classification![7:7]"
PDAL: filters.range: Invalid dimension name in 'limits' option: 'Classification'.

[INFO]    running pdal translate -i "/var/www/data/d2e7a62a-9227-48f5-8e3f-1249d6588fb9/odm_filterpoints/parts/part_214.ply" -o "/var/www/data/d2e7a62a-9227-48f5-8e3f-1249d6588fb9/odm_filterpoints/parts/part_214_filtered.ply" sample range --writers.ply.sized_types=false --writers.ply.storage_mode="little endian" --writers.ply.dims="x=float,y=float,z=float,nx=float,ny=float,nz=float,red=uchar,blue=uchar,green=uchar,views=uchar" --filters.sample.radius=0.01 --filters.range.limits="Classification![7:7]"
PDAL: filters.range: Invalid dimension name in 'limits' option: 'Classification'.

[INFO]    running pdal translate -i "/var/www/data/d2e7a62a-9227-48f5-8e3f-1249d6588fb9/odm_filterpoints/parts/part_179.ply" -o "/var/www/data/d2e7a62a-9227-48f5-8e3f-1249d6588fb9/odm_filterpoints/parts/part_179_filtered.ply" sample range --writers.ply.sized_types=false --writers.ply.storage_mode="little endian" --writers.ply.dims="x=float,y=float,z=float,nx=float,ny=float,nz=float,red=uchar,blue=uchar,green=uchar,views=uchar" --filters.sample.radius=0.01 --filters.range.limits="Classification![7:7]"
PDAL: filters.range: Invalid dimension name in 'limits' option: 'Classification'.

[INFO]    running pdal translate -i "/var/www/data/d2e7a62a-9227-48f5-8e3f-1249d6588fb9/odm_filterpoints/parts/part_113.ply" -o "/var/www/data/d2e7a62a-9227-48f5-8e3f-1249d6588fb9/odm_filterpoints/parts/part_113_filtered.ply" sample range --writers.ply.sized_types=false --writers.ply.storage_mode="little endian" --writers.ply.dims="x=float,y=float,z=float,nx=float,ny=float,nz=float,red=uchar,blue=uchar,green=uchar,views=uchar" --filters.sample.radius=0.01 --filters.range.limits="Classification![7:7]"
PDAL: filters.range: Invalid dimension name in 'limits' option: 'Classification'.

PDAL: filters.range: Invalid dimension name in 'limits' option: 'Classification'.

[INFO]    running pdal translate -i "/var/www/data/d2e7a62a-9227-48f5-8e3f-1249d6588fb9/odm_filterpoints/parts/part_41.ply" -o "/var/www/data/d2e7a62a-9227-48f5-8e3f-1249d6588fb9/odm_filterpoints/parts/part_41_filtered.ply" sample range --writers.ply.sized_types=false --writers.ply.storage_mode="little endian" --writers.ply.dims="x=float,y=float,z=float,nx=float,ny=float,nz=float,red=uchar,blue=uchar,green=uchar,views=uchar" --filters.sample.radius=0.01 --filters.range.limits="Classification![7:7]"
[INFO]    running pdal translate -i "/var/www/data/d2e7a62a-9227-48f5-8e3f-1249d6588fb9/odm_filterpoints/parts/part_187.ply" -o "/var/www/data/d2e7a62a-9227-48f5-8e3f-1249d6588fb9/odm_filterpoints/parts/part_187_filtered.ply" sample range --writers.ply.sized_types=false --writers.ply.storage_mode="little endian" --writers.ply.dims="x=float,y=float,z=float,nx=float,ny=float,nz=float,red=uchar,blue=uchar,green=uchar,views=uchar" --filters.sample.radius=0.01 --filters.range.limits="Classification![7:7]"
PDAL: filters.range: Invalid dimension name in 'limits' option: 'Classification'.

[INFO]    running pdal translate -i "/var/www/data/d2e7a62a-9227-48f5-8e3f-1249d6588fb9/odm_filterpoints/parts/part_102.ply" -o "/var/www/data/d2e7a62a-9227-48f5-8e3f-1249d6588fb9/odm_filterpoints/parts/part_102_filtered.ply" sample range --writers.ply.sized_types=false --writers.ply.storage_mode="little endian" --writers.ply.dims="x=float,y=float,z=float,nx=float,ny=float,nz=float,red=uchar,blue=uchar,green=uchar,views=uchar" --filters.sample.radius=0.01 --filters.range.limits="Classification![7:7]"
[INFO]    running pdal translate -i "/var/www/data/d2e7a62a-9227-48f5-8e3f-1249d6588fb9/odm_filterpoints/parts/part_19.ply" -o "/var/www/data/d2e7a62a-9227-48f5-8e3f-1249d6588fb9/odm_filterpoints/parts/part_19_filtered.ply" sample range --writers.ply.sized_types=false --writers.ply.storage_mode="little endian" --writers.ply.dims="x=float,y=float,z=float,nx=float,ny=float,nz=float,red=uchar,blue=uchar,green=uchar,views=uchar" --filters.sample.radius=0.01 --filters.range.limits="Classification![7:7]"
PDAL: filters.range: Invalid dimension name in 'limits' option: 'Classification'.

[INFO]    running pdal translate -i "/var/www/data/d2e7a62a-9227-48f5-8e3f-1249d6588fb9/odm_filterpoints/parts/part_107.ply" -o "/var/www/data/d2e7a62a-9227-48f5-8e3f-1249d6588fb9/odm_filterpoints/parts/part_107_filtered.ply" sample range --writers.ply.sized_types=false --writers.ply.storage_mode="little endian" --writers.ply.dims="x=float,y=float,z=float,nx=float,ny=float,nz=float,red=uchar,blue=uchar,green=uchar,views=uchar" --filters.sample.radius=0.01 --filters.range.limits="Classification![7:7]"
PDAL: filters.range: Invalid dimension name in 'limits' option: 'Classification'.

PDAL: filters.range: Invalid dimension name in 'limits' option: 'Classification'.

[INFO]    running pdal translate -i "/var/www/data/d2e7a62a-9227-48f5-8e3f-1249d6588fb9/odm_filterpoints/parts/part_1.ply" -o "/var/www/data/d2e7a62a-9227-48f5-8e3f-1249d6588fb9/odm_filterpoints/parts/part_1_filtered.ply" sample range --writers.ply.sized_types=false --writers.ply.storage_mode="little endian" --writers.ply.dims="x=float,y=float,z=float,nx=float,ny=float,nz=float,red=uchar,blue=uchar,green=uchar,views=uchar" --filters.sample.radius=0.01 --filters.range.limits="Classification![7:7]"
PDAL: filters.range: Invalid dimension name in 'limits' option: 'Classification'.

[INFO]    running pdal translate -i "/var/www/data/d2e7a62a-9227-48f5-8e3f-1249d6588fb9/odm_filterpoints/parts/part_39.ply" -o "/var/www/data/d2e7a62a-9227-48f5-8e3f-1249d6588fb9/odm_filterpoints/parts/part_39_filtered.ply" sample range --writers.ply.sized_types=false --writers.ply.storage_mode="little endian" --writers.ply.dims="x=float,y=float,z=float,nx=float,ny=float,nz=float,red=uchar,blue=uchar,green=uchar,views=uchar" --filters.sample.radius=0.01 --filters.range.limits="Classification![7:7]"
PDAL: filters.range: Invalid dimension name in 'limits' option: 'Classification'.

[INFO]    running pdal translate -i "/var/www/data/d2e7a62a-9227-48f5-8e3f-1249d6588fb9/odm_filterpoints/parts/part_200.ply" -o "/var/www/data/d2e7a62a-9227-48f5-8e3f-1249d6588fb9/odm_filterpoints/parts/part_200_filtered.ply" sample range --writers.ply.sized_types=false --writers.ply.storage_mode="little endian" --writers.ply.dims="x=float,y=float,z=float,nx=float,ny=float,nz=float,red=uchar,blue=uchar,green=uchar,views=uchar" --filters.sample.radius=0.01 --filters.range.limits="Classification![7:7]"

pierotofy · 13 September 2021 13:43

Oh interesting. Does it work if you remove pc-classify? Might be a bug.

Saijin_Naib · 13 September 2021 13:45

Yep, my thoughts exactly.

Re-trying now. I’ll let you know in like, 10 hours when my freaking internet finishes the upload Our local fiber-to-the-premises ISP can’t roll out here fast enough.

Anecdotal, but I don’t recall having this with v2.5.7 when processing locally this week against some of the same datasets, and I always have the --pc-classify option.

pierotofy · 13 September 2021 16:18

Possibly related (different error though): Fix bug when pc-filter is zero by pierotofy · Pull Request #1346 · OpenDroneMap/ODM · GitHub

pierotofy · 13 September 2021 16:19

Ok, I can reproduce it with any dataset with:;

--pc-classify --pc-filter 0 --pc-sample 0.01

Fix coming soon…

Saijin_Naib · 13 September 2021 16:22

Sweet, thanks for the fix.

When do you think it’ll propagate to Lightning?

pierotofy · 13 September 2021 16:25

1-2 days.

system · 13 October 2021 16:25

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.