Hi everyone,
I have been experiencing this issue with WebODM + Ligthning. Assets in the “all.zip” file seems to be ok but it shows th same behavior even if i try to manually import to WebODM.
here is part of the Task Log in WebODM:
100 - done.
[INFO] Finished odm_orthophoto stage
[INFO] Running odm_report stage
[INFO] Wrote /var/www/data/1cc2c70a-b033-4047-b734-90b35d478148/odm_report/shots.geojson
[INFO] Finished odm_report stage
[INFO] No more stages to run
[INFO] MMMMMMMMMMMNNNMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMNNNMMMMMMMMMMM
[INFO] MMMMMMdo:…—…/sNMMMMMMMMMMMMMMMMMMMMMMMMMMNs/…—…:odMMMMMM
[INFO] MMMMy-.odNMMMMMNy//mMMMMMMMMMMMMMMMMMMMMMMm//hNMMMMMNdo.-yMMMM
[INFO] MMN/sMMMMMMMMMNNMm/yMMMMMMMMMMMMMMMMMMMMy/mMNNMMMMMMMMNs/MMM
[INFO] MM/ hMMMMMMMMNs.+MMM/ dMMMMMMMMMMMMMMMMMMh +MMM+.sNMMMMMMMMh +MM
[INFO] MN /MMMMMMNo/./mMMMMN :MMMMMMMMMMMMMMMMMM: NMMMMm/./oNMMMMMM: NM
[INFO] Mm +MMMMMN+ /MMMMMMM-MMMMMMMMMMMMMMMMMM-MMMMMMM: oNMMMMM+ mM
[INFO] MM…NMMNs./mNMMMMMMMy sMMMMMMMMMMMMMMMMMMo hMMMMMMMNm/.sNMMN-MM [INFO] MMd:mMNomMMMMMMMMMy:MMMMMMMNmmmmNMMMMMMN:hMMMMMMMMMdoNMm-dMM [INFO] MMMm:.omMMMMMMMMNh/ sdmmho/.…-``-/sddh+ /hNMMMMMMMMdo.:mMMM [INFO] MMMMMd+--/osss+:-:/ :- .ym+ hmo``:-` `+:-:ossso/-:+dMMMMM [INFO] MMMMMMMNmhysosydmNMo /ds`/NMM+ hMMd..dh. sMNmdysosyhmNMMMMMMM [INFO] MMMMMMMMMMMMMMMMMMMs .:-:``hmmN+ yNmds -:.:`-NMMMMMMMMMMMMMMMMMM [INFO] MMMMMMMMMMMMMMMMMMN.-mNm- //:::. -:://: +mMd`-NMMMMMMMMMMMMMMMMM [INFO] MMMMMMMMMMMMMMMMMM+ dMMN -MMNNN+ yNNNMN :MMMs sMMMMMMMMMMMMMMMMM [INFO] MMMMMMMMMMMMMMMMMM`.mmmy /mmmmm/ smmmmm``mmmh :MMMMMMMMMMMMMMMMM [INFO] MMMMMMMMMMMMMMMMMM``:::- ./////. -:::::` :::: -MMMMMMMMMMMMMMMMM [INFO] MMMMMMMMMMMMMMMMMM:`mNNd /NNNNN+ hNNNNN .NNNy +MMMMMMMMMMMMMMMMM [INFO] MMMMMMMMMMMMMMMMMMd`/MMM.`ys+//. -/+oso +MMN.`mMMMMMMMMMMMMMMMMM [INFO] MMMMMMMMMMMMMMMMMMMy /o:- `oyhd/ shys+ `-:s-`hMMMMMMMMMMMMMMMMMM [INFO] MMMMMMMMNmdhhhdmNMMM` +d+ sMMM+ hMMN:`hh- sMMNmdhhhdmNMMMMMMMM [INFO] MMMMMms:::/++//::+ho .+- /dM+ hNh- +/` -h+:://++/::/smMMMMM [INFO] MMMN+./hmMMMMMMNds- ./oso:.``:. :-``.:os+- -sdNMMMMMMmy:.oNMMM [INFO] MMm-.hMNhNMMMMMMMMNo`/MMMMMNdhyyyyhhdNMMMM+`oNMMMMMMMMNhNMh.-mMM [INFO] MM:`mMMN/-sNNMMMMMMMo yMMMMMMMMMMMMMMMMMMy sMMMMMMMNNs-/NMMm`:MM [INFO] Mm /MMMMMd/.-oMMMMMMN :MMMMMMMMMMMMMMMMMM-`MMMMMMMo-./dMMMMM/ NM [INFO] Mm /MMMMMMm:-`sNMMMMN :MMMMMMMMMMMMMMMMMM-`MMMMMNs`-/NMMMMMM/ NM [INFO] MM:`mMMMMMMMMd/-sMMMo yMMMMMMMMMMMMMMMMMMy sMMMs-/dMMMMMMMMd`:MM [INFO] MMm-.hMMMMMMMMMdhMNo`+MMMMMMMMMMMMMMMMMMMM+`oNMhdMMMMMMMMMh.-mMM [INFO] MMMNo./hmNMMMMMNms--yMMMMMMMMMMMMMMMMMMMMMMy--smNMMMMMNmy/.oNMMM [INFO] MMMMMms:-:/+++/:-+hMMMMMMMMMMMMMMMMMMMMMMMMMNh+-:/+++/:-:smMMMMM [INFO] MMMMMMMMNdhhyhdmMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMmdhyhhmNMMMMMMMM [INFO] MMMMMMMMMMMMMMMNNNNNMMMMMMNNNNNNMMMMMMMMNNMMMMMMMNNMMMMMMMMMMMMM [INFO] MMMMMMMMMMMMMh/-...-+dMMMm......:+hMMMMs../MMMMMo..sMMMMMMMMMMMM [INFO] MMMMMMMMMMMM/ /yhy- sMMm -hhy/ :NMM+ oMMMy /MMMMMMMMMMMM [INFO] MMMMMMMMMMMy /MMMMN` NMm /MMMMo +MM: .` yMd :MMMMMMMMMMMM
[INFO] MMMMMMMMMMM+ sMMMMM: hMm /MMMMd -MM- /s h.d- -MMMMMMMMMMMM
[INFO] MMMMMMMMMMMs +MMMMM. mMm /MMMMy /MM. +M/ yM: MMMMMMMMMMMM [INFO] MMMMMMMMMMMN- smNm/ +MMm :NNdo .mMMoMM+/yMM/ MMMMMMMMMMMM [INFO] MMMMMMMMMMMMNo- :yMMMm :sNMMM sMMMMMMM+ NMMMMMMMMMMM
[INFO] MMMMMMMMMMMMMMMNmmNMMMMMMMNmmmmNMMMMMMMNNMMMMMMMMMNNMMMMMMMMMMMM
[INFO] ODM app finished - Wed Dec 02 20:57:23 2020
Postprocessing: /var/www/data/1cc2c70a-b033-4047-b734-90b35d478148
Found point cloud: odm_georeferencing/odm_georeferenced_model.laz
Scanning input
1/1: odm_georeferencing/odm_georeferenced_model.laz
Determined SRS from an input file
Entwine Version: 2.1.0
EPT Version: 1.0.0
Input:
File: odm_georeferencing/odm_georeferenced_model.laz
Total points: 44,138,153
Density estimate (per square unit): 53.3268
Threads: [1, 7]
Output:
Path: entwine_pointcloud/
Data type: laszip
Hierarchy type: json
Sleep count: 2,097,152
Scale: 0.01
Offset: (314633, 3044915, 230)
Metadata:
SRS: EPSG:32614
Bounds: [(314166, 3044470, 199), (315099, 3045360, 261)]
Cube: [(314165, 3044447, -238), (315101, 3045383, 698)]
Storing dimensions: [
X:int32, Y:int32, Z:int32, Intensity:uint16, ReturnNumber:uint8,
NumberOfReturns:uint8, ScanDirectionFlag:uint8,
EdgeOfFlightLine:uint8, Classification:uint8, ScanAngleRank:float,
UserData:uint8, PointSourceId:uint16, GpsTime:double, Red:uint16,
Green:uint16, Blue:uint16, OriginId:uint32
]
Build parameters:
Span: 128
Resolution 2D: 128 * 128 = 16,384
Resolution 3D: 128 * 128 * 128 = 2,097,152
Maximum node size: 65,536
Minimum node size: 16,384
Cache size: 64
Adding 0 - odm_georeferencing/odm_georeferenced_model.laz
Pushes complete - joining…
It seems to be doing something in the background, this is how it looks in the terminal.
Docker also whows “worker” running a process
and this is part of the container log
$ docker logs 64973e017a4b | tail -20
Postgres is up - executing command
wait-for-it.sh: waiting for broker:6379 without a timeout
wait-for-it.sh: broker:6379 is available after 0 seconds
wait-for-it.sh: waiting for webapp:8000 without a timeout
wait-for-it.sh: webapp:8000 is available after 34 seconds
INFO Initializing GRASS engine using /usr/bin/grass78
/usr/local/lib/python3.6/dist-packages/celery/platforms.py:801: RuntimeWarning: You’re running the worker with superuser privileges: this is
absolutely not recommended!
Please specify a different user using the --uid option.
User information: uid=0 euid=0 gid=0 egid=0
uid=uid, euid=euid, gid=gid, egid=egid,
INFO Canceling Task [Task of 2020-12-01T03:55:07.791Z] (3d20395c-1d4a-4934-a628-e6dbc60e6742)
[2020-12-01 14:46:27,634: INFO/ForkPoolWorker-2] worker.tasks.process_task[f12de5fc-749c-4423-bd34-80eb2df94f92]: Canceling Task [Task of 2020-12-01T03:55:07.791Z] (3d20395c-1d4a-4934-a628-e6dbc60e6742)
INFO Removing Task [Task of 2020-12-01T03:55:07.791Z] (3d20395c-1d4a-4934-a628-e6dbc60e6742)
[2020-12-01 15:02:50,805: INFO/ForkPoolWorker-4] worker.tasks.process_task[54ef552b-2589-4ed4-a397-b99d4d76238b]: Removing Task [Task of 2020-12-01T03:55:07.791Z] (3d20395c-1d4a-4934-a628-e6dbc60e6742)
INFO Processing… Task [Task of 2020-12-01T15:02:56.306Z] (9fcd7fe8-0829-4098-9670-7f46d21a41bc)
[2020-12-01 15:12:26,907: INFO/ForkPoolWorker-5] worker.tasks.process_task[e9035122-ce8e-4cee-bda8-f4b9dcb0d650]: Processing… Task [Task of 2020-12-01T15:02:56.306Z] (9fcd7fe8-0829-4098-9670-7f46d21a41bc)
WARNING Task [Task of 2020-12-01T15:02:56.306Z] (9fcd7fe8-0829-4098-9670-7f46d21a41bc) connection/timeout error: HTTPSConnectionPool(host=‘spark1.webodm.net’, port=443): Max retries exceeded with url: /info?token=xxxxxx (Caused by SSLError(SSLError(“bad handshake: SysCallError(104, ‘ECONNRESET’)”,),)). We’ll try reprocessing at the next tick.
[2020-12-01 15:18:18,927: WARNING/ForkPoolWorker-14] worker.tasks.process_task[76713fd4-772d-4bee-a310-2e2c8ae0d2b0]: Task [Task of 2020-12-01T15:02:56.306Z] (9fcd7fe8-0829-4098-9670-7f46d21a41bc) connection/timeout error: HTTPSConnectionPool(host=‘spark1.webodm.net’, port=443): Max retries exceeded with url: /info?token=xxxxxx (Caused by SSLError(SSLError(“bad handshake: SysCallError(104, ‘ECONNRESET’)”,),)). We’ll try reprocessing at the next tick.
WARNING Task [Task of 2020-12-01T15:02:56.306Z] (9fcd7fe8-0829-4098-9670-7f46d21a41bc) connection/timeout error: HTTPSConnectionPool(host=‘spark1.webodm.net’, port=443): Max retries exceeded with url: /task/98126bfb-a1aa-4ce8-9a5a-b97f253211d9/info?with_output=5567&token=xxxxxx (Caused by SSLError(SSLError(“bad handshake: SysCallError(104, ‘ECONNRESET’)”,),)). We’ll try reprocessing at the next tick.
[2020-12-01 15:48:38,935: WARNING/ForkPoolWorker-51] worker.tasks.process_task[8f834261-30e7-4ebb-bfae-4e386e04af56]: Task [Task of 2020-12-01T15:02:56.306Z] (9fcd7fe8-0829-4098-9670-7f46d21a41bc) connection/timeout error: HTTPSConnectionPool(host=‘spark1.webodm.net’, port=443): Max retries exceeded with url: /task/98126bfb-a1aa-4ce8-9a5a-b97f253211d9/info?with_output=5567&token=xxxxxx (Caused by SSLError(SSLError(“bad handshake: SysCallError(104, ‘ECONNRESET’)”,),)). We’ll try reprocessing at the next tick.
WARNING Task [Task of 2020-12-01T15:02:56.306Z] (9fcd7fe8-0829-4098-9670-7f46d21a41bc) connection/timeout error: HTTPSConnectionPool(host=‘spark1.webodm.net’, port=443): Max retries exceeded with url: /task/98126bfb-a1aa-4ce8-9a5a-b97f253211d9/info?with_output=5808&token=xxxxxx (Caused by NewConnectionError(’<urllib3.connection.VerifiedHTTPSConnection object at 0x7f5d09587a58>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution’,)). We’ll try reprocessing at the next tick.
[2020-12-01 15:58:04,621: WARNING/ForkPoolWorker-64] worker.tasks.process_task[bd34b8bc-d478-4ba0-8085-ead338443fa7]: Task [Task of 2020-12-01T15:02:56.306Z] (9fcd7fe8-0829-4098-9670-7f46d21a41bc) connection/timeout error: HTTPSConnectionPool(host=‘spark1.webodm.net’, port=443): Max retries exceeded with url: /task/98126bfb-a1aa-4ce8-9a5a-b97f253211d9/info?with_output=5808&token=xxxxxx (Caused by NewConnectionError(’<urllib3.connection.VerifiedHTTPSConnection object at 0x7f5d09587a58>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution’,)). We’ll try reprocessing at the next tick.
WARNING Task [Task of 2020-12-01T15:02:56.306Z] (9fcd7fe8-0829-4098-9670-7f46d21a41bc) connection/timeout error: HTTPSConnectionPool(host=‘spark1.webodm.net’, port=443): Max retries exceeded with url: /task/98126bfb-a1aa-4ce8-9a5a-b97f253211d9/info?with_output=5808&token=xxxxxx (Caused by NewConnectionError(’<urllib3.connection.VerifiedHTTPSConnection object at 0x7f5d0955b048>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution’,)). We’ll try reprocessing at the next tick.
[2020-12-01 15:58:18,981: WARNING/ForkPoolWorker-63] worker.tasks.process_task[a54f79ff-b66c-46e8-8b8a-b43f53cec476]: Task [Task of 2020-12-01T15:02:56.306Z] (9fcd7fe8-0829-4098-9670-7f46d21a41bc) connection/timeout error: HTTPSConnectionPool(host=‘spark1.webodm.net’, port=443): Max retries exceeded with url: /task/98126bfb-a1aa-4ce8-9a5a-b97f253211d9/info?with_output=5808&token=xxxxxx (Caused by NewConnectionError(’<urllib3.connection.VerifiedHTTPSConnection object at 0x7f5d0955b048>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution’,)). We’ll try reprocessing at the next tick.
WARNING Task [Task of 2020-12-01T15:02:56.306Z] (9fcd7fe8-0829-4098-9670-7f46d21a41bc) connection/timeout error: HTTPSConnectionPool(host=‘spark1.webodm.net’, port=443): Max retries exceeded with url: /task/98126bfb-a1aa-4ce8-9a5a-b97f253211d9/info?with_output=5808&token=xxxxxx (Caused by NewConnectionError(’<urllib3.connection.VerifiedHTTPSConnection object at 0x7f5d09557198>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution’,)). We’ll try reprocessing at the next tick.
[2020-12-01 15:58:33,013: WARNING/ForkPoolWorker-63] worker.tasks.process_task[09434ab1-e2a3-496c-9bf7-51e81c8c2f19]: Task [Task of 2020-12-01T15:02:56.306Z] (9fcd7fe8-0829-4098-9670-7f46d21a41bc) connection/timeout error: HTTPSConnectionPool(host=‘spark1.webodm.net’, port=443): Max retries exceeded with url: /task/98126bfb-a1aa-4ce8-9a5a-b97f253211d9/info?with_output=5808&token=xxxxxx (Caused by NewConnectionError(’<urllib3.connection.VerifiedHTTPSConnection object at 0x7f5d09557198>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution’,)). We’ll try reprocessing at the next tick.
WARNING Task [Task of 2020-12-01T15:02:56.306Z] (9fcd7fe8-0829-4098-9670-7f46d21a41bc) connection/timeout error: HTTPSConnectionPool(host=‘spark1.webodm.net’, port=443): Max retries exceeded with url: /task/98126bfb-a1aa-4ce8-9a5a-b97f253211d9/info?with_output=5808&token=xxxxxx (Caused by NewConnectionError(’<urllib3.connection.VerifiedHTTPSConnection object at 0x7f5d09557630>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution’,)). We’ll try reprocessing at the next tick.
[2020-12-01 15:58:47,361: WARNING/ForkPoolWorker-63] worker.tasks.process_task[e8d75f0e-eb49-40a7-9f74-2fc44779a8ed]: Task [Task of 2020-12-01T15:02:56.306Z] (9fcd7fe8-0829-4098-9670-7f46d21a41bc) connection/timeout error: HTTPSConnectionPool(host=‘spark1.webodm.net’, port=443): Max retries exceeded with url: /task/98126bfb-a1aa-4ce8-9a5a-b97f253211d9/info?with_output=5808&token=xxxxxx (Caused by NewConnectionError(’<urllib3.connection.VerifiedHTTPSConnection object at 0x7f5d09557630>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution’,)). We’ll try reprocessing at the next tick.
WARNING Task [Task of 2020-12-01T15:02:56.306Z] (9fcd7fe8-0829-4098-9670-7f46d21a41bc) connection/timeout error: HTTPSConnectionPool(host=‘spark1.webodm.net’, port=443): Read timed out. (read timeout=30). We’ll try reprocessing at the next tick.
[2020-12-01 17:05:03,909: WARNING/ForkPoolWorker-132] worker.tasks.process_task[49982c44-b817-4e36-8179-eeb34ec02152]: Task [Task of 2020-12-01T15:02:56.306Z] (9fcd7fe8-0829-4098-9670-7f46d21a41bc) connection/timeout error: HTTPSConnectionPool(host=‘spark1.webodm.net’, port=443): Read timed out. (read timeout=30). We’ll try reprocessing at the next tick.
WARNING Task [Task of 2020-12-01T15:02:56.306Z] (9fcd7fe8-0829-4098-9670-7f46d21a41bc) connection/timeout error: (‘Connection aborted.’, OSError("(104, ‘ECONNRESET’)",)). We’ll try reprocessing at the next tick.
[2020-12-01 17:45:19,394: WARNING/ForkPoolWorker-137] worker.tasks.process_task[9624f1cb-a99f-440d-8e52-1f18ca6b78b1]: Task [Task of 2020-12-01T15:02:56.306Z] (9fcd7fe8-0829-4098-9670-7f46d21a41bc) connection/timeout error: (‘Connection aborted.’, OSError("(104, ‘ECONNRESET’)",)). We’ll try reprocessing at the next tick.
INFO Processing status: 40 for Task [Task of 2020-12-01T15:02:56.306Z] (9fcd7fe8-0829-4098-9670-7f46d21a41bc)
[2020-12-01 18:07:46,471: INFO/ForkPoolWorker-189] worker.tasks.process_task[79f9a704-f585-41df-8820-6bd41a165291]: Processing status: 40 for Task [Task of 2020-12-01T15:02:56.306Z] (9fcd7fe8-0829-4098-9670-7f46d21a41bc)
INFO Downloading all.zip for Task [Task of 2020-12-01T15:02:56.306Z] (9fcd7fe8-0829-4098-9670-7f46d21a41bc)
[2020-12-01 18:07:46,476: INFO/ForkPoolWorker-189] worker.tasks.process_task[79f9a704-f585-41df-8820-6bd41a165291]: Downloading all.zip for Task [Task of 2020-12-01T15:02:56.306Z] (9fcd7fe8-0829-4098-9670-7f46d21a41bc)
INFO Cleaned up: tmpe1h1o2y3.tif (1606769218.2503695)
[2020-12-01 21:46:22,476: INFO/ForkPoolWorker-625] worker.tasks.cleanup_tmp_directory[198781bc-8af0-4bd1-b34b-f2a84968c0e5]: Cleaned up: tmpe1h1o2y3.tif (1606769218.2503695)
INFO Cleaned up: tmpswjzluoq_cogeo.tif (1606769218.1983695)
[2020-12-01 21:46:22,522: INFO/ForkPoolWorker-625] worker.tasks.cleanup_tmp_directory[198781bc-8af0-4bd1-b34b-f2a84968c0e5]: Cleaned up: tmpswjzluoq_cogeo.tif (1606769218.1983695)
INFO Cleaned up: tmpwzi3wbmh.upload (1606780626.0994487)
[2020-12-02 00:46:22,566: INFO/ForkPoolWorker-1003] worker.tasks.cleanup_tmp_directory[7daf820b-7f84-44f9-bd09-37289f48e4f7]: Cleaned up: tmpwzi3wbmh.upload (1606780626.0994487)
INFO Cleaned up: tmpzof_maaq_cogeo.tif (1606781061.971602)
[2020-12-02 00:46:22,617: INFO/ForkPoolWorker-1003] worker.tasks.cleanup_tmp_directory[7daf820b-7f84-44f9-bd09-37289f48e4f7]: Cleaned up: tmpzof_maaq_cogeo.tif (1606781061.971602)
INFO Cleaned up: tmp3gvsegd5.tif (1606781061.9756021)
[2020-12-02 00:46:22,739: INFO/ForkPoolWorker-1003] worker.tasks.cleanup_tmp_directory[7daf820b-7f84-44f9-bd09-37289f48e4f7]: Cleaned up: tmp3gvsegd5.tif (1606781061.9756021)
INFO Cleaned up: tmpo59rdsti.tif (1606796203.085183)
[2020-12-02 04:46:23,320: INFO/ForkPoolWorker-1483] worker.tasks.cleanup_tmp_directory[c0cf3e47-7b8f-46da-ac7c-e7fe258416d6]: Cleaned up: tmpo59rdsti.tif (1606796203.085183)
INFO Cleaned up: tmpbq7z62ng.upload (1606795409.0546894)
[2020-12-02 04:46:23,511: INFO/ForkPoolWorker-1483] worker.tasks.cleanup_tmp_directory[c0cf3e47-7b8f-46da-ac7c-e7fe258416d6]: Cleaned up: tmpbq7z62ng.upload (1606795409.0546894)
INFO Cleaned up: tmpqjc__c3z_cogeo.tif (1606796203.0811827)
[2020-12-02 04:46:23,847: INFO/ForkPoolWorker-1483] worker.tasks.cleanup_tmp_directory[c0cf3e47-7b8f-46da-ac7c-e7fe258416d6]: Cleaned up: tmpqjc__c3z_cogeo.tif (1606796203.0811827)
INFO Processing… Task [TSO 24OCT2020 L2 NO_GCP] (3fc99c08-f601-431d-a114-ee5f51515b6b)
[2020-12-02 18:10:52,520: INFO/ForkPoolWorker-3075] worker.tasks.process_task[a1a87f63-9a95-478f-9d9e-f77e6a3abd92]: Processing… Task [TSO 24OCT2020 L2 NO_GCP] (3fc99c08-f601-431d-a114-ee5f51515b6b)
WARNING Task [TSO 24OCT2020 L2 NO_GCP] (3fc99c08-f601-431d-a114-ee5f51515b6b) connection/timeout error: HTTPSConnectionPool(host=‘spark1.webodm.net’, port=443): Max retries exceeded with url: /task/1cc2c70a-b033-4047-b734-90b35d478148/info?with_output=6369&token=xxxxxx (Caused by ConnectTimeoutError(<urllib3.connection.VerifiedHTTPSConnection object at 0x7f5d09556f28>, ‘Connection to spark1.webodm.net timed out. (connect timeout=30)’)). We’ll try reprocessing at the next tick.
[2020-12-02 19:29:27,110: WARNING/ForkPoolWorker-3305] worker.tasks.process_task[1fc7dd4b-80db-4fdd-a35d-f2215a9be1a2]: Task [TSO 24OCT2020 L2 NO_GCP] (3fc99c08-f601-431d-a114-ee5f51515b6b) connection/timeout error: HTTPSConnectionPool(host=‘spark1.webodm.net’, port=443): Max retries exceeded with url: /task/1cc2c70a-b033-4047-b734-90b35d478148/info?with_output=6369&token=xxxxxx (Caused by ConnectTimeoutError(<urllib3.connection.VerifiedHTTPSConnection object at 0x7f5d09556f28>, ‘Connection to spark1.webodm.net timed out. (connect timeout=30)’)). We’ll try reprocessing at the next tick.
WARNING Task [TSO 24OCT2020 L2 NO_GCP] (3fc99c08-f601-431d-a114-ee5f51515b6b) connection/timeout error: HTTPSConnectionPool(host=‘spark1.webodm.net’, port=443): Max retries exceeded with url: /task/1cc2c70a-b033-4047-b734-90b35d478148/info?with_output=7604&token=xxxxxx (Caused by ConnectTimeoutError(<urllib3.connection.VerifiedHTTPSConnection object at 0x7f5d09513d30>, ‘Connection to spark1.webodm.net timed out. (connect timeout=30)’)). We’ll try reprocessing at the next tick.
[2020-12-02 20:21:33,029: WARNING/ForkPoolWorker-3464] worker.tasks.process_task[b6aa389f-c8f5-4ca1-aded-0de4cce8daa4]: Task [TSO 24OCT2020 L2 NO_GCP] (3fc99c08-f601-431d-a114-ee5f51515b6b) connection/timeout error: HTTPSConnectionPool(host=‘spark1.webodm.net’, port=443): Max retries exceeded with url: /task/1cc2c70a-b033-4047-b734-90b35d478148/info?with_output=7604&token=xxxxxx (Caused by ConnectTimeoutError(<urllib3.connection.VerifiedHTTPSConnection object at 0x7f5d09513d30>, ‘Connection to spark1.webodm.net timed out. (connect timeout=30)’)). We’ll try reprocessing at the next tick.
INFO Processing status: 40 for Task [TSO 24OCT2020 L2 NO_GCP] (3fc99c08-f601-431d-a114-ee5f51515b6b)
[2020-12-02 21:01:24,979: INFO/ForkPoolWorker-3582] worker.tasks.process_task[b178df51-cc54-4004-aef6-b273cae1c925]: Processing status: 40 for Task [TSO 24OCT2020 L2 NO_GCP] (3fc99c08-f601-431d-a114-ee5f51515b6b)
INFO Downloading all.zip for Task [TSO 24OCT2020 L2 NO_GCP] (3fc99c08-f601-431d-a114-ee5f51515b6b)
[2020-12-02 21:01:24,979: INFO/ForkPoolWorker-3582] worker.tasks.process_task[b178df51-cc54-4004-aef6-b273cae1c925]: Downloading all.zip for Task [TSO 24OCT2020 L2 NO_GCP] (3fc99c08-f601-431d-a114-ee5f51515b6b)
INFO Extracting all.zip for Task [TSO 24OCT2020 L2 NO_GCP] (3fc99c08-f601-431d-a114-ee5f51515b6b)
[2020-12-02 21:05:16,761: INFO/ForkPoolWorker-3582] worker.tasks.process_task[b178df51-cc54-4004-aef6-b273cae1c925]: Extracting all.zip for Task [TSO 24OCT2020 L2 NO_GCP] (3fc99c08-f601-431d-a114-ee5f51515b6b)
INFO Extracted all.zip for Task [TSO 24OCT2020 L2 NO_GCP] (3fc99c08-f601-431d-a114-ee5f51515b6b)
[2020-12-02 21:06:46,622: INFO/ForkPoolWorker-3582] worker.tasks.process_task[b178df51-cc54-4004-aef6-b273cae1c925]: Extracted all.zip for Task [TSO 24OCT2020 L2 NO_GCP] (3fc99c08-f601-431d-a114-ee5f51515b6b)
INFO Optimizing /webodm/app/media/project/2/task/3fc99c08-f601-431d-a114-ee5f51515b6b/assets/odm_orthophoto/odm_orthophoto.tif as Cloud Optimized GeoTIFF
[2020-12-02 21:06:47,177: INFO/ForkPoolWorker-3582] worker.tasks.process_task[b178df51-cc54-4004-aef6-b273cae1c925]: Optimizing /webodm/app/media/project/2/task/3fc99c08-f601-431d-a114-ee5f51515b6b/assets/odm_orthophoto/odm_orthophoto.tif as Cloud Optimized GeoTIFF
WARNING Using legacy implementation (GDAL >= 3.1 not found)
[2020-12-02 21:06:50,084: WARNING/ForkPoolWorker-3582] worker.tasks.process_task[b178df51-cc54-4004-aef6-b273cae1c925]: Using legacy implementation (GDAL >= 3.1 not found)
Checking for celery… OK
Starting worker using broker at redis://broker