Unable to use Shared access signatures - Not enough supported images in /var/www/data/8c5a9f40-29c3-4e12-abf5-d73cb7a093c6/images

General Help PyODM
1

Hello,

Just want to state that I find this project awesome and I am going to soon share what I built with it!

I have run into a weird error, I send my pictures through pyODM to the node by specifying a list with url:s to a blob/bucket where my images reside. This have worked great, however I dont want to leave my pictures on public access so I added the url with a SAS:

Grant limited access to data with shared access signatures (SAS) - Azure Storage | Microsoft Docs

Essentially a key on the end of the url which grant access to the user/machine. For some reason this breaks my processing. I can clearly see that the images are uploaded and not empty because it takes some time. I ran into following error 1 sec into the processing(it says images:29 which is all):

[INFO]    use_3dmesh: False
[INFO]    use_exif: False
[INFO]    use_fixed_camera_params: False
[INFO]    use_hybrid_bundle_adjustment: False
[INFO]    verbose: False
[INFO]    ==============
[INFO]    Running dataset stage
[INFO]    Loading dataset from: /var/www/data/8c5a9f40-29c3-4e12-abf5-d73cb7a093c6/images
[ERROR]   Not enough supported images in /var/www/data/8c5a9f40-29c3-4e12-abf5-d73cb7a093c6/images

Does anyone have a clue? I am totally dumbfounded about this!

Best Regards,
Rasmus

2 Likes
2

Is it possible that the images are being named with the SAS appended to them or something?

3

That is actually an interesting point yes! Probably the reason, hmm, cant think of any work arounds for this one.

4

Can you list the contents of that directory after upload? What’s in there?

5

Actually never figure it out instead i am using a file share and mounted it! Works faster actually!

2 Likes
6

Hello, I have hit the same issue and you are correct. When the images are copied to the local NodeODM filesystem, the access token appended to the URL is propagated to the filename.

In my case, I am working with pre-signed URLs from s3 with the structure of <http/https filepath>.jpg?<aws credentials>. The images are saved with a filename like image.jpg?<aws credentials>. There is no file extension so NodeODM does not recognize the files as supported images.

This is actually an easy fix. I’ve added a check to see if an image path has an access token appended to the URL and to parse the correct filename accordingly.

This has worked successfully with both s3 pre-signed URLs and Azure blog SaS. I would like to open a pull request to contribute this to PyODM if there is interest.

1 Like
7

A pull request would be most welcome! :+1:

2 Likes
8

Pull request created :slight_smile:

2 Likes