Webodm / Docker image file size

dlf · 19 July 2018 14:15

Hello guys

First of all, thanks for this great project. Im new to ODM and Docker. I managed to get WebODM running on my 2012 Macbook pro retina with 16GB Ram. I set in Docker the disk image location on a external SSD with a file size of 64GB. And I set the Docker resources to 7 CPUs, 13 GB Memory, 1 GB Swap (what the heck is Swap??)
Please let me know if I should improve those settings.
I was really surprised when I noticed the docker image file size of 10GB after the installation of WebODM. Before installing it was over 2GB. This is by far the largest (in GB) software I have ever used. Is this normal?

Afrer I made the fist project with 160 pictures which didnt give me a good result yet. This increased the file size to 16GB I deleted the project but the file size remained. I tried a new project right after, but it just kept increasing the docker image to 20GB. What the hell is that?? Guys, I dont have unlimited resources to waste. How can I ‘really’ delete the projects from the Docker container?

Thanks in advance for the replys

pierotofy · 19 July 2018 16:37

Hey @dlf, your allocation of resources seem good as it is.

Swap is a paging storage mechanism: Memory paging - Wikipedia (if you have enough RAM, allocated, you need not to worry about it).

I suggest you familiarize with how docker works, as WebODM runs on top of it.

https://getintodevops.com/blog/keeping-the-whale-happy-how-to-clean-up-after-docker

dlf · 19 July 2018 17:14

Hi @pierotofy . Thanks a lot for your fast reply. I appreciate that a lot. Ok i will study that article.
Do you know if we can expect to have ODM natively with an Interface on OSX any time in the near future?

pierotofy · 19 July 2018 17:35

Running natively is a possibility, an OSX interface… likely not to happen for a while. But we welcome contributions, so perhaps somebody will work on one.

dlf · 20 July 2018 08:39

@pierotofy I checked your link:
https://getintodevops.com/blog/keeping-the-whale-happy-how-to-clean-up-after-docker
but i cant seem to figure out how to wipe the ODM project data from the container to reduce the file size back to webodm factory setting.
Would it be possible to get into the container with a ftp program like cyberduck and remove files manually? Im not so much into coding and deleting projects in browser/webodm doesnt seem to do anything.

dlf · 20 July 2018 09:33

there is another issue i ran into. I made a Task with 24 images, which worked fine with a completed task. So i did another task with 166 images. The task couldnt be completed and gave me an “Process exited with code 1” message. I even set the disk image size of docker to 112GB, but it didnt help. How can I fix this problem?

dlf · 20 July 2018 11:00

I tried with 91 images and the same error happened.

dlf · 20 July 2018 15:33

even with 65 images i get the Error. What could be the issue?

2018-07-20 11:51:43,709 INFO: DJI_0364.JPG resection inliers: 0 / 8
2018-07-20 11:51:43,732 INFO: Some images can not be added
2018-07-20 11:51:43,733 INFO: -------------------------------------------------------
2018-07-20 11:51:43,799 DEBUG: Ceres Solver Report: Iterations: -2, Initial cost: -1.000000e+00, Final cost: -1.000000e+00, Termination: FAILURE
2018-07-20 11:51:43,801 INFO: Removed outliers: 0
Traceback (most recent call last):
File “/code/SuperBuild/src/opensfm/bin/opensfm”, line 34, in
command.run(args)
File “/code/SuperBuild/src/opensfm/opensfm/commands/reconstruct.py”, line 21, in run
report = reconstruction.incremental_reconstruction(data)
File “/code/SuperBuild/src/opensfm/opensfm/reconstruction.py”, line 1134, in incremental_reconstruction
data, graph, reconstruction, remaining_images, gcp)
File “/code/SuperBuild/src/opensfm/opensfm/reconstruction.py”, line 1096, in grow_reconstruction
align_reconstruction(reconstruction, gcp, config)
File “/code/SuperBuild/src/opensfm/opensfm/align.py”, line 17, in align_reconstruction
res = align_reconstruction_similarity(reconstruction, gcp, config)
File “/code/SuperBuild/src/opensfm/opensfm/align.py”, line 59, in align_reconstruction_similarity
return align_reconstruction_naive_similarity(reconstruction, gcp)
File “/code/SuperBuild/src/opensfm/opensfm/align.py”, line 83, in align_reconstruction_naive_similarity
T = tf.superimposition_matrix(X.T, Xp.T, scale=True)
File “/code/SuperBuild/src/opensfm/opensfm/transformations.py”, line 1041, in superimposition_matrix
scale=scale, usesvd=usesvd)
File “/code/SuperBuild/src/opensfm/opensfm/transformations.py”, line 954, in affine_matrix_from_points
u, s, vh = numpy.linalg.svd(numpy.dot(v1, v0.T))
File “/usr/local/lib/python2.7/dist-packages/numpy/linalg/linalg.py”, line 1444, in svd
u, s, vh = gufunc(a, signature=signature, extobj=extobj)
File “/usr/local/lib/python2.7/dist-packages/numpy/linalg/linalg.py”, line 98, in _raise_linalgerror_svd_nonconvergence
raise LinAlgError(“SVD did not converge”)
numpy.linalg.linalg.LinAlgError: SVD did not converge
Traceback (most recent call last):
File “/code/run.py”, line 47, in
plasm.execute(niter=1)
File “/code/scripts/run_opensfm.py”, line 134, in process
(context.pyopencv_path, context.opensfm_path, tree.opensfm))
File “/code/opendm/system.py”, line 34, in run
raise Exception(“Child returned {}”.format(retcode))
Exception: Child returned 1

dlf · 21 July 2018 06:57

I cant get more than 30 images calculated without that “Process exited with Code 1” error. Please somebody help me fix this. I’m desperate. My goal is to be able to do 500 for a high res obj and orthophoto.
Here is my specs:

Is there any other information needed to fix this?
Please dont tell me my laptop is too old. I dont mind to have my computer running overnight for the calculation.

dsvbr2 · 21 July 2018 15:57

I’m running into the same problem. I just can’t run no more than 88 images. Any thoughts?

dlf · 23 July 2018 12:19

@dsvbr2 I managed to process 139 images sucessdully with another set of footage. Im guessing ODM is very sensitive to overlap. But on the other hand it cant handle too many either because ODM uses up too much Ram. Whether you get a result or not depends a lot on your luck. please correct me if im wrong.

I tried to do 171 images. Everything seemed to be fine for 6 hours until i ran into Error code 137

dsvbr2 · 24 July 2018 00:46

Now I could managed to process 151 images in lower resolution (approximately 1 megapixel). I’m overlapping 55%, how much are you overlapping your photos?

dlf · 24 July 2018 09:23

@dsvbr2 Arent you you worried you will loose precision of your result if you use low res images?
How can i know the precentage of the overlap?

dsvbr2 · 27 July 2018 16:25

@dlf yes, but I just did this to see if I can run webodm. The percentage of the overlap you can configure it in your planned mission (in the app).

dlf · 2 August 2018 09:08

@dsvbr2 Which app are you using? I am using pix4dcapture with dji Spark. I cant set there any percentage only distance between pictures

dsvbr2 · 4 August 2018 13:17

I’ve been used MapPilot and Pix4d Capture too.

smathermather · 6 August 2018 00:46

I use DroneDeploy for the capture. It’s not perfect (there’s one additional setting I’d like that it doesn’t have — camera angle) but works very nicely for DJI. You can set percentage overlap in the app, and then offload and process in ODM.