Does the tile in Cesium Ion button still work?

RussellGrew · 27 April 2020 20:40

Dear all,

Thanks for making WebODM available. It is great.

The tile in cesium ion button used to work for me. Now it hangs:

I was wondering if anyone else has experienced the same?

I can curl "https://api.cesium.com/v1/assets" -H "Authorization: Bearer <your_access_token>" fine from the webapp container where the photos go. I have also tried creating and saving a new token.

Thanks.

pierotofy · 27 April 2020 22:28

@brandon-cesium might know if something has changed?

RussellGrew · 27 April 2020 23:47

I do note that if I run the API curl test using a particular windows build I get

curl: (60) SSL certificate problem: self signed certificate in certificate chain
More details here: https://curl.haxx.se/docs/sslcerts.html

curl failed to verify the legitimacy of the server and therefore could not
establish a secure connection to it. To learn more about this situation and
how to fix it, please visit the web page mentioned above.

for curl -V

curl 7.60.0 (x86_64-pc-win32) libcurl/7.60.0 OpenSSL/1.0.2j zlib/1.2.8 WinIDN
Release-Date: 2018-05-16
Protocols: dict file ftp ftps gopher http https imap imaps ldap pop3 pop3s rtsp smb smbs smtp smtps telnet tftp
Features: AsynchDNS IDN IPv6 Largefile SSPI Kerberos SPNEGO NTLM SSL libz HTTPS-proxy

but the Git Bash curl correctly returns the list of assets and has Version info

curl 7.60.0 (x86_64-w64-mingw32) libcurl/7.60.0 OpenSSL/1.0.2o (WinSSL) zlib/1.2.11 libidn2/2.0.5 nghttp2/1.32.0
Release-Date: 2018-05-16
Protocols: dict file ftp ftps gopher http https imap imaps ldap ldaps pop3 pop3s rtsp smtp smtps telnet tftp
Features: AsynchDNS IDN IPv6 Largefile SSPI Kerberos SPNEGO NTLM SSL libz TLS-SRP HTTP2 HTTPS-proxy MultiSSL Metalink

This may be unrelated. I never explored it prior to having issues.

OmarShehata · 12 May 2020 13:53

@pierotofy running locally, I can see that the reason the Cesium ion plugin is failing is because it seems like the task fails to launch. Here’s the console output I see:

worker      | [2020-05-12 13:43:25,652: ERROR/MainProcess] Received unregistered task of type 'plugins.cesium-ion.api_views.upload_to_ion'.
worker      | The message has been ignored and discarded.
worker      | 
worker      | Did you remember to import the module containing this task?
worker      | Or maybe you're using relative imports?
worker      | 
worker      | Please see
worker      | http://docs.celeryq.org/en/latest/internals/protocol.html
worker      | for more information.
worker      | 
worker      | The full contents of the message body was:
worker      | b'[["84e9b336-4841-4388-a29f-d2e88649499c", "POINTCLOUD", "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJqdGkiOiIyZjI5YjBkMS04ZDFlLTQzOTctOWRmYy02NWE5M2Y2MmQ4NTUiLCJpZCI6MTYyMywic2NvcGVzIjpbImFzbCIsImFzciIsImFzdyJdLCJpYXQiOjE1NjM0NzY5MDl9.xVpx6vhlcS_hZvUxWMjuGB3zmajcGKKlS3TFx-vpJeE", "/webodm/app/media/project/1/task/84e9b336-4841-4388-a29f-d2e88649499c/assets/odm_georeferencing/odm_georeferenced_model.laz", "First Project | Task of 2019-11-03T18:01:06.004Z \\u2060\\u2014 Pointcloud", "", "", {"baseTerrainId": "", "textureFormat": false}], {}, {"callbacks": null, "errbacks": null, "chain": null, "chord": null}]' (608b)
worker      | Traceback (most recent call last):
worker      |   File "/usr/local/lib/python3.6/site-packages/celery/worker/consumer/consumer.py", line 559, in on_task_received
worker      |     strategy = strategies[type_]
worker      | KeyError: 'plugins.cesium-ion.api_views.upload_to_ion'

But this task is registered here:

github.com

OpenDroneMap/WebODM/blob/bdf9fcfebfabcf7f4825962ff92139449d8939ad/plugins/cesium-ion/api_views.py#L360


      
                  if self._logger is not None and progress - self._last_log > self._log_step_size:
                      self._logger.info(f"Upload progress: {progress * 100}%")
                      self._last_log = progress
          
          
        set_asset_info(self._task_id, self._asset_type, self._asset_info)
          
          

          
# Arg order is very important for task deconstruction.
          # If order is changed make sure that the refresh API call is updated
          @task
          def upload_to_ion(
              task_id,
              asset_type,
              token,
              asset_path,
              name,
              description="",
              attribution="",
              options={},
          ):
              asset_logger = LoggerAdapter(prefix=f"Task {task_id} {asset_type}", logger=logger)

And the console output here implies it successfully registers the plugin:

webapp      | INFO Initializing GRASS engine using /usr/bin/grass78
webapp      | INFO Booting WebODM 1.3.6
webapp      | INFO Registered [plugins.contours.plugin]
webapp      | INFO Registered [plugins.cesium-ion.plugin]
webapp      | INFO Registered [plugins.lightning.plugin]
webapp      | INFO Registered [plugins.posm-gcpi.plugin]
webapp      | INFO Registered [plugins.diagnostic.plugin]
webapp      | INFO Registered [plugins.osm-quickedit.plugin]
webapp      | INFO Registered [plugins.fullscreen.plugin]
webapp      | INFO Registered [plugins.openaerialmap.plugin]
webapp      | INFO Registered [plugins.cloudimport.plugin]
webapp      | INFO Registered [plugins.measure.plugin]

So I’m not sure why it’s failing to find the task. Did something change recently in the way tasks must be registered?

pierotofy · 12 May 2020 14:01

I’m not really sure; I don’t maintain the Cesium plugin, but it’s possible something broke. We’d welcome pull requests to fix it.

OmarShehata · 21 May 2020 01:50

I’m happy to submit a pull request, but I’m not sure why this is failing. It doesn’t seem like WebODM’s plugin architecture has changed recently. I also just tried it with a clean install on Windows, and it worked. Destroyed all containers and volumes back on my Linux and tried again, same Received unregistered task of type 'plugins.cesium-ion.api_views.upload_to_ion' error.

@RussellGrew what OS are you testing on?

RussellGrew · 21 May 2020 03:35

Thanks Omar.

I am running windows 10 with linux docker containers. I seem to recall selecting this when installing docker originally, though the latest version doesn’t seem to have this as a choice.

Following your rinse and repeat experience with linux I do note that for me maybe the plugin stopped working after a ./webodm.sh update. I did not remove the Docker images and my existing WebODM projects were retained.

This isn’t a massive issue for us presently. Though I do note it gets a mention on your website WebODM – Cesium

For comparison I will take a look using linux.

RussellGrew · 22 May 2020 06:54

Following up, I get the same error as Omar on both Windows and Linux. The Linux install is completely new ~ it is a new PC.

I do wonder if Plugins refactoring by pierotofy · Pull Request #846 · OpenDroneMap/WebODM · GitHub is somehow involved. Though I am more end-user and campaigner than developer.

I have tried disabling/enabling the plugin via the settings panel, and also uploading all.zip as a fresh project and attempting to tile that per instructions WebODM/plugins/cesium-ion at bdf9fcfebfabcf7f4825962ff92139449d8939ad · OpenDroneMap/WebODM · GitHub

Happy Friday to all.

ryyiann · 22 July 2020 16:08

I have the same problem that Russel is mentioning that Omar originally posted error logs for.

I am working on a Mac.

ODM to Cesium was really cool when it worked. Now i’m stuck. It would be great if someone found out why this plugin is failing.

nchamo · 26 July 2020 21:31

Hey everyone!

Another plugin was having the same problem with Celery, and @pierotofy came up with a solution that should help all plugins (check the PR). If anyone here is tech-savy, you might be able to implement the same fix for the Cesium Ion plugin.

Just wanted to mention that

ryyiann · 10 August 2020 14:49

This looks like a solution but I am struggling to understand how to implement it for the cesium plugin

nchamo · 11 August 2020 13:18

Hey @ryyiann!

I might be wrong, but I think that you will need to

Remove the @task annotation in https://github.com/OpenDroneMap/WebODM/blob/master/plugins/cesium-ion/api_views.py#L359
Replace the line upload_to_ion.delay( (https://github.com/OpenDroneMap/WebODM/blob/master/plugins/cesium-ion/api_views.py#L266) with run_function_async(upload_to_ion,
Replace the line from app.plugins.worker import task (https://github.com/OpenDroneMap/WebODM/blob/master/plugins/cesium-ion/api_views.py#L11) with from app.plugins.worker import run_function_async

Hope that helps!

ryyiann · 18 August 2020 15:56

Thank you for the suggestion!

I tried this and still no success with the cesium ion plugin.

mateo · 16 March 2021 03:51

I got the cesium-ion plugin working again. 2 caveats:

It doesn’t place 3d models properly (othos and DEMs are perfect). I’ve used my own datasets as well as samples from ODM github to no avail; 3d models always end up within 500m of where they should be just not exact. Adjust Location feature in Cesium after the upload is tedious but not difficult. I’m willing to try other datasets if someone has one that was known to work before the plugin broke.
If you use Cloud Import for a project, the Cesium plugin buttons do not appear at all. Other projects without Cloud Import still function properly.

Is it better to do a pull request now to share the code changes or get these bugs worked out first?

Saijin_Naib · 16 March 2021 04:05

Mateo, welcome!

Awesome work getting the plugin mostly-working again!

I’m not the authoritative answer on this, but I don’t personally see the harm of setting up a draft PR now, so Piero and Stephen can see what you’ve got going, and maybe offer some feedback.

mateo · 16 March 2021 05:23

Thanks!

I’ve got a neat idea to extend it a little further. Cesium lets you store original data on S3 to optimize quota use (blog post). We could definitely get the plugin to first upload to S3 and then link to Cesium for processing and tile storage.

Do I need anyone’s permission to implement that or just full steam ahead?

Saijin_Naib · 16 March 2021 05:42

Hmm… I think if you document that feature and let people easily configure /disable it, I can’t see anything wrong with it.

Again, not the authoritative voice on this.

Seems like a great feature, though.

pierotofy · 16 March 2021 13:30

Up to you!

Could be an interesting addition! I wouldn’t remove the option to upload directly to Cesium though (if that was the intent?), not everyone wants to use S3.

mateo · 17 March 2021 00:46

I’m thinking a separate plugin for upload to S3 and then an option to link the two. I’ll start a separate thread in the dev forum.

mateo · 19 March 2021 17:26

Pull Request

Notes:

3d models do not import with correct location yet - they are pretty close but not perfect.
Tile button does not appear with Cloud Import enabled (only on projects that actually use cloud import)