Space in ODM Install Path

bernarde · 17 December 2021 10:42

Hi all,

I stumble on a new problem today. ODM was not working on one of my colleague computer. After investigation it is because the install path of ODM contains a space. We can not change the install path of ODM and I have found that the issue is already reported on github :

github.com/OpenDroneMap/ODM

ODM Console not working when installed in Program Files

opened 07:18AM - 28 Jul 21 UTC

closed 11:29PM - 23 Dec 21 UTC

casbolwerk

bug help wanted

### How did you install OpenDroneMap? (Docker, natively, ...)? The native rel…eases download on windows ### What's your browser and operating system? (Copy/paste the output of https://www.whatismybrowser.com/) Firefox 90 on windows 10 ### What is the problem? The pyvenv.cfg file (and possibly others) are not created when ODM is installed in a path with a space in it. The issue lies with the win32env.bat file, which is supposed to create this pyvenv.cfg file. It does not handle paths with spaces and ends up not creating it ### What should be the expected behavior? If this is a feature request, please describe in detail the changes you think should be made to the code, citing files and lines where changes should be made, if possible. win32env.bat should be able to handle spaces in paths. It goes wrong at the two lines 24 and 25 in win32env.bat. ``` echo home = %ODMBASE%\python38> %PYENVCFG% echo include-system-site-packages = false>> %PYENVCFG% ``` I tried putting quotation marks around the paths like `"%PYENVCFG%"`, but while creating the pyvenv.cfg file it still got errors because the python code could not handle the spaces in `%ODMBASE%\python38`. This might be a bigger issue than just creating the pyvenv.cfg file. ### How can we reproduce this? (What steps did you do to trigger the problem? What parameters are you using for processing? If possible please include a copy of your dataset uploaded on Google Drive or Dropbox. Be detailed) Install ODM inside a path with a space (Program Files is a popular one on windows) and try to run it. It will say `No pyvenv.cfg file` in the console. Then go ahead and change win32env.bat.

I looked at the win32env.bat script and know that there will be several thing to change in order to make it work. My question is the following, do you think that fixing this script only will do it or will it be a never ending fix leading me to explore the python files ?

Thank you

bernarde · 17 December 2021 15:37

I managed to correct various files, win32.bat, opendm/osfm.py and stages/openmvs.py

All seems to process correctly and I can retrieve the output files until it crashed during report generation because it can not find the proj.db.

ERROR 1: PROJ: proj_create_from_database: Cannot find proj.db

I do not understand why it can not found proj.db during report phase whereas it found it during the rest of the computations.

set PROJ_LIB="%GDALBASE%\data\proj"

Any idea or help ?

Saijin_Naib · 17 December 2021 16:20

Maybe how you’re escaping the path is not working properly for the last example when expanding the path variable?

pierotofy · 17 December 2021 18:48

Can you post the full error message / stack trace?

Also, interested in opening a pull request on GitHub to help us fix this for everyone?

bernarde · 20 December 2021 07:48

Hi,
Here is a more complete log :

[INFO]    Running odm_report stage
[INFO]    Exporting shots.geojson
ERROR 1: PROJ: proj_create_from_database: Cannot find proj.db
ERROR 1: PROJ: proj_create_from_database: Cannot find proj.db
ERROR 1: PROJ: proj_create: unrecognized format / unknown name
ERROR 6: Cannot find coordinate operations from `' to `'
Traceback (most recent call last):
  File "C:\space folder\ODM\\run.py", line 53, in <module>
    retcode = app.execute()
  File "C:\space folder\ODM\stages\odm_app.py", line 130, in execute
    raise e
  File "C:\space folder\ODM\stages\odm_app.py", line 94, in execute
    self.first_stage.run()
  File "C:\space folder\ODM\opendm\types.py", line 346, in run
    self.next_stage.run(outputs)
  File "C:\space folder\ODM\opendm\types.py", line 346, in run
    self.next_stage.run(outputs)
  File "C:\space folder\ODM\opendm\types.py", line 346, in run
    self.next_stage.run(outputs)
  [Previous line repeated 8 more times]
  File "C:\space folder\ODM\opendm\types.py", line 327, in run
    self.process(self.args, outputs)
  File "C:\space folder\ODM\stages\odm_report.py", line 55, in process
    shots = get_geojson_shots_from_opensfm(tree.opensfm_reconstruction, pseudo_geotiff=tree.odm_orthophoto_tif)
  File "C:\space folder\ODM\opendm\shots.py", line 75, in get_geojson_shots_from_opensfm
    trans_coords = crstrans.TransformPoint(utm_coords[0], utm_coords[1], utm_coords[2])
  File "C:\space folder\ODM\venv\lib\site-packages\osgeo\osr.py", line 1088, in TransformPoint
    return _osr.CoordinateTransformation_TransformPoint(self, *args)
NotImplementedError: Wrong number or type of arguments for overloaded function 'CoordinateTransformation_TransformPoint'.
  Possible C/C++ prototypes are:
    OSRCoordinateTransformationShadow::TransformPoint(double [3])
    OSRCoordinateTransformationShadow::TransformPoint(double [4])
    OSRCoordinateTransformationShadow::TransformPoint(double [3],double,double,double)
    OSRCoordinateTransformationShadow::TransformPoint(double [4],double,double,double,double)

Of course once it seems fix for me I will have a look at the rules for pull request and share with everyone

pierotofy · 20 December 2021 16:20

So strange. This error ERROR 1: PROJ: proj_create_from_database: Cannot find proj.db only gets shown at the report stage?

I can’t think of a reason why PROJ would be complaining, since we set PROJ_LIB before launching the python process that runs the report code…

Are you running your tests with the --rerun-all flag?

bernarde · 20 December 2021 22:18

I am totally not re-running with the “–rerun-all” flag. I will try and give you more info tomorrow.

bernarde · 21 December 2021 08:08

Ok, this makes no sense to me neither but I can confirm that I obtain the same error log by both using the --re-run flag or by deleting the content of the folder.

bernarde · 22 December 2021 13:14

Ok, I have found that it was a case specific.

Without gps data in picture it process to the repport stage but if there is gps data it crashes with the same type of error log during the run_opensfm stage. I will work on this and let you now as I progess.

[INFO]    Altitude data detected, enabling it for GPS alignment
[INFO]    ['use_exif_size: no', 'flann_algorithm: KDTREE', 'feature_process_size: 2304', 'feature_min_frames: 8000', 'processes: 8', 'matching_gps_neighbors: 8', 'matching_gps_distance: 0', 'optimize_camera_parameters: yes', 'undistorted_image_format: tif', 'bundle_outlier_filtering_type: AUTO', 'sift_peak_threshold: 0.066', 'align_orientation_prior: vertical', 'triangulation_type: ROBUST', 'retriangulation_ratio: 2', 'matcher_type: FLANN', 'feature_type: SIFT', 'use_altitude_tag: yes', 'align_method: auto', 'local_bundle_radius: 0']
ERROR 1: PROJ: proj_create_from_database: Cannot find proj.db
ERROR 1: PROJ: proj_create_from_database: Cannot find proj.db
ERROR 1: PROJ: proj_create: unrecognized format / unknown name
ERROR 6: Cannot find coordinate operations from `' to `'
Traceback (most recent call last):
  File "C:\space folder\ODM\\run.py", line 53, in <module>
    retcode = app.execute()
  File "C:\space folder\ODM\stages\odm_app.py", line 130, in execute
    raise e
  File "C:\space folder\ODM\stages\odm_app.py", line 94, in execute
    self.first_stage.run()
  File "C:\space folder\ODM\opendm\types.py", line 346, in run
    self.next_stage.run(outputs)
  File "C:\space folder\ODM\opendm\types.py", line 346, in run
    self.next_stage.run(outputs)
  File "C:\space folder\ODM\opendm\types.py", line 346, in run
    self.next_stage.run(outputs)
  File "C:\space folder\ODM\opendm\types.py", line 327, in run
    self.process(self.args, outputs)
  File "C:\space folder\ODM\stages\run_opensfm.py", line 32, in process
    octx.setup(args, tree.dataset_raw, reconstruction=reconstruction, rerun=self.rerun())
  File "C:\space folder\ODM\opendm\osfm.py", line 271, in setup
    self.write_reference_lla(reconstruction.georef.utm_east_offset, reconstruction.georef.utm_north_offset, reconstruction.georef.proj4())
  File "C:\space folder\ODM\opendm\osfm.py", line 450, in write_reference_lla
    lon, lat = location.transform2(CRS.from_proj4(proj4), longlat, offset_x, offset_y)
  File "C:\space folder\ODM\opendm\location.py", line 60, in transform2
    return transformer(from_srs, to_srs).TransformPoint(x, y, 0)[:2]
  File "C:\space folder\ODM\venv\lib\site-packages\osgeo\osr.py", line 1091, in TransformPoint
    return _osr.CoordinateTransformation_TransformPoint(self, *args)
NotImplementedError: Wrong number or type of arguments for overloaded function 'CoordinateTransformation_TransformPoint'.
  Possible C/C++ prototypes are:
    OSRCoordinateTransformationShadow::TransformPoint(double [3])
    OSRCoordinateTransformationShadow::TransformPoint(double [4])
    OSRCoordinateTransformationShadow::TransformPoint(double [3],double,double,double)
    OSRCoordinateTransformationShadow::TransformPoint(double [4],double,double,double,double)

pierotofy · 22 December 2021 15:08

Yes the error is definitely PROJ related.

PROJ_LIB controls the path to the proj.db database.

For some reason, it doesn’t like spaces in its current form.

bernarde · 22 December 2021 16:32

Yes here is my current knowledge :

PROJ_LIB we set in win32_env.bat is not necessarily the one use by _VALIDATED_PROJ_DATA in datadir.py. In my case it is the internal_datadir that is used and whose value is : ODM\venv\lib\site-packages\pyproj\proj_dir\share\proj
I have test setting PROJ_LIB with surrounding quotes which led to the previous error log.
Strangely without surrounding quotes in PROJ_LIB it seems to work both for georeferenced pictures and not georeferenced pictures.

Everything seems to work it my test cases. I have made a pull request :

Please can someone test it on his side ?

system · 21 January 2022 16:32

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.