Processing complete did not upload to server

Message below was all I got…

Uploading 1 files to S3…

Cannot upload file to S3: SlowDown, retrying… 0 Cannot upload file to S3: TimeoutError, retrying… 1 Cannot upload file to S3: ServiceUnavailable, retrying… 2 Cannot upload file to S3: ServiceUnavailable, retrying… 3 Cannot upload file to S3: SlowDown, retrying… 4 Cannot upload file to S3: SlowDown, retrying… 5 Cannot upload file to S3: SlowDown, retrying… 6

1 Like

Mm, I just processed a task via webdom.net and it worked OK. I wonder if this was due to some maintenance on the cloud storage provider’s part.

Try again and let us know if it happens again?

2 Likes

Thanks I am trying a different dataset, lots of time uploading …is there any way to recover the original or is all lost?

Thanks for your help

lk

webodm.net does not keep track of uploads, so I’m afraid you will need to re-upload :confused:

1 Like

Oh well, it is on the way…I will report the results Thank you

1 Like

looks like the same thing is happening…different dataset
1089 images the only parameter i changed from the default is the las export

image

Yup failed…is there anything that can help with this the error message is attached.
I can download the processing report and send it along…I am now trying a batch with less that 1000 photos and without thw PCLAS button clicked.

Your task [elwood split 2] could not complete. Here’s a copy of the error message:

[INFO] MMMMMMMMMMMMMMMMMMN.-mNm- //:::. -:://: +mMd-NMMMMMMMMMMMMMMMMM [INFO] MMMMMMMMMMMMMMMMMM+ dMMN -MMNNN+ yNNNMN :MMMs sMMMMMMMMMMMMMMMMM [INFO] MMMMMMMMMMMMMMMMMM.mmmy /mmmmm/ smmmmmmmmh :MMMMMMMMMMMMMMMMM [INFO] MMMMMMMMMMMMMMMMMM:::- ./////. -::::: :::: -MMMMMMMMMMMMMMMMM [INFO] MMMMMMMMMMMMMMMMMM:mNNd /NNNNN+ hNNNNN .NNNy +MMMMMMMMMMMMMMMMM
[INFO] MMMMMMMMMMMMMMMMMMd/MMM.ys+//. -/+oso +MMN.mMMMMMMMMMMMMMMMMM [INFO] MMMMMMMMMMMMMMMMMMMy /o:- oyhd/ shys+ -:s-hMMMMMMMMMMMMMMMMMM
[INFO] MMMMMMMMNmdhhhdmNMMM +d+ sMMM+ hMMN:hh- sMMNmdhhhdmNMMMMMMMM
[INFO] MMMMMms:::/++//::+ho .± /dM+ hNh- +/ -h+:://++/::/smMMMMM [INFO] MMMN+./hmMMMMMMNds- ./oso:.``:. :-``.:os+- -sdNMMMMMMmy:.oNMMM [INFO] MMm-.hMNhNMMMMMMMMNo/MMMMMNdhyyyyhhdNMMMM+oNMMMMMMMMNhNMh.-mMM [INFO] MM:mMMN/-sNNMMMMMMMo yMMMMMMMMMMMMMMMMMMy sMMMMMMMNNs-/NMMm:MM [INFO] Mm /MMMMMd/.-oMMMMMMN :MMMMMMMMMMMMMMMMMM-MMMMMMMo-./dMMMMM/ NM
[INFO] Mm /MMMMMMm:-sNMMMMN :MMMMMMMMMMMMMMMMMM-MMMMMNs-/NMMMMMM/ NM [INFO] MM:mMMMMMMMMd/-sMMMo yMMMMMMMMMMMMMMMMMMy sMMMs-/dMMMMMMMMd:MM [INFO] MMm-.hMMMMMMMMMdhMNo+MMMMMMMMMMMMMMMMMMMM+oNMhdMMMMMMMMMh.-mMM [INFO] MMMNo./hmNMMMMMNms--yMMMMMMMMMMMMMMMMMMMMMMy--smNMMMMMNmy/.oNMMM [INFO] MMMMMms:-:/+++/:-+hMMMMMMMMMMMMMMMMMMMMMMMMMNh+-:/+++/:-:smMMMMM [INFO] MMMMMMMMNdhhyhdmMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMmdhyhhmNMMMMMMMM [INFO] MMMMMMMMMMMMMMMNNNNNMMMMMMNNNNNNMMMMMMMMNNMMMMMMMNNMMMMMMMMMMMMM [INFO] MMMMMMMMMMMMMh/-...-+dMMMm......:+hMMMMs../MMMMMo..sMMMMMMMMMMMM [INFO] MMMMMMMMMMMM/ /yhy- sMMm -hhy/ :NMM+ oMMMy /MMMMMMMMMMMM [INFO] MMMMMMMMMMMy /MMMMN NMm /MMMMo +MM: .yMd``` :MMMMMMMMMMMM [INFO] MMMMMMMMMMM+ sMMMMM: hMm /MMMMd -MM- /sh.d- -MMMMMMMMMMMM [INFO] MMMMMMMMMMMs +MMMMM. mMm /MMMMy /MM. +M/ yM: MMMMMMMMMMMM
[INFO] MMMMMMMMMMMN- smNm/ +MMm :NNdo .mMM oMM+/yMM/ MMMMMMMMMMMM
[INFO] MMMMMMMMMMMMNo- :yMMMm :sNMMM` sMMMMMMM+ NMMMMMMMMMMM
[INFO] MMMMMMMMMMMMMMMNmmNMMMMMMMNmmmmNMMMMMMMNNMMMMMMMMMNNMMMMMMMMMMMM
[INFO] ODM app finished - Sun Nov 01 12:18:04 2020
/usr/local/lib/python3.6/dist-packages/sklearn/externals/joblib/externals/cloudpickle/cloudpickle.py:47: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module’s documentation for alternative uses import imp Compressing all.zip

Uploading 1 files to S3…
Cannot upload file to S3: SlowDown, retrying… 0 Cannot upload file to S3: SlowDown, retrying… 1 Cannot upload file to S3: TimeoutError, retrying… 2 Cannot upload file to S3: TimeoutError, retrying… 3 Cannot upload file to S3: TimeoutError, retrying… 4 Cannot upload file to S3: TimeoutError, retrying… 5 Cannot upload file to S3: TimeoutError, retrying… 6

I loaded a tast with less than 1000 images and I also did not ask for PCLAS allzip is 5.1GB and I am downloading it now.
Is there a problem with tasks larger that 1000 photos?

Thanks for your help with this.
lk

{“uuid”:“f20733bb-4464-400c-a016-fe770264ce7c”,“name”:“split1-3”,“dateCreated”:1604279175176,“processingTime”:25315812,“status”:{“code”:40},“options”:[{“name”:“pc-sample”,“value”:0.01},{“name”:“dsm”,“value”:true},{“name”:“dem-resolution”,“value”:2},{“name”:“orthophoto-resolution”,“value”:2},{“name”:“optimize-disk-space”,“value”:true}],“imagesCount”:965,“progress”:100}

1 Like

Yup, so now it shows me a completed task, 5.1 GB in a all.zip file.
trying to download, works fine at a good speed reporting 1 hour remaining or thereabouts, then when it reaches 1.5gb down it stalls and will not resume…I have tried numerous times same result.
any hints on this?
thank you
lk

If you use a download accelerator, are you able to download it faster? https://sourceforge.net/projects/urlget/

It might be due to the larger file size. I will investigate this and report back.

2 Likes

I was able to download at a site in the city, I am on country LTE broadband, and was able to download on a fibre hookup. It did stall several times but I was able to get the file …
I will try the accelerator on the next one I have 5000 photos to go…

Thank you

1 Like

I’ve also made some changes to the software, which should fix the original issue you reported about uploads to S3 failing: https://github.com/OpenDroneMap/NodeODM/pull/132

Could you retry one last time to process the dataset that was giving you troubles?

3 Likes

Thanks very much, have loaded and processes a dataset with over 1000 photos, the results are in and everything was very good, I did click on PCLAS so the compressed file was 7.5gb.

I did load up a download accelerator (IDM) and it does help although I am having trouble on slower connections with files larger than 4gb it will get close to done and then have a timeout error.
I am able to download in the city on a fibre connection…

thank you for all your help, I am working on a faster connection at my home but is slow coming…
The upload portions are no trouble it just takes time.

1 Like

If you skip PCLAS it will help a lot on download size: default is LAZ which is orders of magnitude smaller. If you need a LAS, you can convert from LAZ on your local machine.

2 Likes

Yep, LAZ is a lossless compressed form of the plain-text LAS. Far nicer to transfer around.

2 Likes

Thanks I will try that
lk