From @sberryman on Fri Mar 09 2018 20:20:15 GMT+0000 (UTC)
FYI ONLY!
I noticed a recent commit to the Dockerfile and was quite interested. Then I started digging in to see what the Dockerfile was actually doing.
There are some glaring issues with the below section based on docker caching of layers. When reviewing the best practices for docker images, They state Always combine RUN apt-get update with apt-get install in the same RUN statement. The reason is that any apt-get install on subsequent layers are using the cached apt-get update from previous layers. Meaning if you re-run docker build on a machine that has already built ODM and donāt make any changes to the layers prior to apt-get update youāll be using old cached lists and installing (potentially) old versions.
Then there is the issue of cleanup of APT. Running cleanup on a separate layer doesnāt remove the intermediate layers which have the cached data. So you either need to remove the cache lists in the same RUN command as you are doing apt-get update and apt-get install or just remove it from the dockerfile as it is useless.
https://github.com/OpenDroneMap/OpenDroneMap/blob/master/Dockerfile#L58-L59
Copied from original issue: https://github.com/OpenDroneMap/OpenDroneMap/issues/781
From @dakotabenjamin on Fri Mar 09 2018 20:33:00 GMT+0000 (UTC)
This used to be the case, Iām not sure why it changed.
From @sberryman on Fri Mar 09 2018 20:38:06 GMT+0000 (UTC)
Sorry, Iām not sure what you mean by āthis used to be the caseā. Are you talking about the removal of APT cache files? Iām pretty sure you only used to be able to get rid of intermediate layers by flattening the image, not very many people do that though.
Update: technically docker calls it -squash so hopefully I didnāt confuse you with āflattenā
From @sberryman on Fri Mar 09 2018 21:33:59 GMT+0000 (UTC)
Any idea why libvtk6-dev header files are getting installed on line 56? They are already included in the massive dependency install phase on line 15.
From @sberryman on Fri Mar 09 2018 21:52:03 GMT+0000 (UTC)
Want me to take a stab at rewriting it? How condensed do you want the final image? We can run almost everything in a single layer or I can try and split it out a bit. It would help to know what you tend to update. Does the build time of the image matter?
I could install the majority of deps near the top and then install build-essential, cmake, perform build and clean up files on another layer.
This would leave you with apt dependencies in the 1st layer, pip dependencies in 2nd, env & copy on a few more layers and finally build and cleanup on the final layer
From @sberryman on Thu Mar 15 2018 14:49:01 GMT+0000 (UTC)
I played around with the Dockerfile a bit, here is docker history for a local build of ODM (96be8c4)
IMAGE CREATED CREATED BY SIZE COMMENT
f4e1f13d0075 Less than a second ago /bin/sh -c #(nop) ENTRYPOINT ["python" "/coā¦ 0B
69e075ebbf62 Less than a second ago /bin/sh -c rm -rf /code/SuperBuild/download ā¦ 0B
8712d7541ebc Less than a second ago /bin/sh -c apt-get clean && rm -rf /var/lib/ā¦ 0B
64bb9330591e Less than a second ago /bin/sh -c apt-get install -y libvtk6-dev 299MB
d6fe276c3611 Less than a second ago /bin/sh -c apt-get -y remove libgl1-mesa-driā¦ 3.23MB
be27339e4c81 Less than a second ago /bin/sh -c cd SuperBuild && mkdir build && cā¦ 2.46GB
8695805db5a7 28 minutes ago /bin/sh -c #(nop) COPY file:d2dd9312ac40f85cā¦ 6B
fd3d44ede059 28 minutes ago /bin/sh -c #(nop) COPY file:7c13e4e876f9f1e8ā¦ 2.31kB
478225373e32 28 minutes ago /bin/sh -c #(nop) COPY file:de9972b7e00aab74ā¦ 3.91kB
77e3f134b1bd 28 minutes ago /bin/sh -c #(nop) COPY dir:0096577dd9215d190ā¦ 18.5kB
4a99d26ffff9 28 minutes ago /bin/sh -c #(nop) COPY dir:d612083a96627f68dā¦ 100kB
8ffc941bc51f 28 minutes ago /bin/sh -c #(nop) COPY file:48c2781c0c3cef57ā¦ 292B
494c9c556426 28 minutes ago /bin/sh -c #(nop) COPY file:91a496b3597da96aā¦ 1.57kB
fc361d40a3e0 28 minutes ago /bin/sh -c #(nop) COPY dir:68c81018f4833e263ā¦ 453kB
38efbc40404a 28 minutes ago /bin/sh -c #(nop) COPY dir:e0e90f8904a9ed21aā¦ 74.8kB
91c2051df250 28 minutes ago /bin/sh -c #(nop) COPY dir:779e102780c9e2f15ā¦ 272kB
a66b4bf113bf 28 minutes ago /bin/sh -c #(nop) COPY file:0ee2b2f83c2c5f46ā¦ 4.91kB
fe3fc6309d03 28 minutes ago /bin/sh -c #(nop) COPY file:449f3516ab95493fā¦ 547B
cb47c53f7025 28 minutes ago /bin/sh -c #(nop) COPY file:2475dfdfd7a6111cā¦ 654B
a3d14dbfe747 28 minutes ago /bin/sh -c #(nop) WORKDIR /code 0B
5035510c7737 28 minutes ago /bin/sh -c mkdir /code 0B
9858aeeed408 28 minutes ago /bin/sh -c #(nop) ENV LD_LIBRARY_PATH=:/codā¦ 0B
53a7d71dc731 28 minutes ago /bin/sh -c #(nop) ENV PYTHONPATH=:/code/Supā¦ 0B
c8ec01695612 28 minutes ago /bin/sh -c #(nop) ENV PYTHONPATH=:/code/Supā¦ 0B
2c91acaea5aa 28 minutes ago /bin/sh -c pip install -U PyYAML exifread gpā¦ 353MB
5a7e8746e9ae 30 minutes ago /bin/sh -c pip install setuptools 2.85MB
96b6e0695ffb 30 minutes ago /bin/sh -c pip install --upgrade pip 8.22MB
40b6d629abeb 30 minutes ago /bin/sh -c apt-get remove libdc1394-22-dev 0B
91f8de57c68f 30 minutes ago /bin/sh -c apt-get install --no-install-recoā¦ 1.67GB
6fd16cdd33fe 36 minutes ago /bin/sh -c apt-get update -y 686kB
d4d0361edc78 36 minutes ago /bin/sh -c add-apt-repository -y ppa:george-ā¦ 6.27kB
076942099b74 36 minutes ago /bin/sh -c add-apt-repository -y ppa:ubuntugā¦ 15.9kB
f6ae1c8f8633 36 minutes ago /bin/sh -c apt-get install software-propertiā¦ 0B
63771a799817 36 minutes ago /bin/sh -c apt-get update -y 43.1MB
d0b382d3989c 5 days ago /bin/sh -c #(nop) ENV DEBIAN_FRONTEND=noninā¦ 0B
166cfc3f6974 6 weeks ago /bin/sh -c #(nop) CMD ["/sbin/my_init"] 0B
<missing> 6 weeks ago /bin/sh -c #(nop) ENV DEBIAN_FRONTEND=teletyā¦ 0B
<missing> 6 weeks ago /bin/sh -c /bd_build/prepare.sh && /bd_builā¦ 97.3MB
<missing> 6 weeks ago /bin/sh -c #(nop) COPY dir:e7a5eda59d878c69cā¦ 39.4kB
<missing> 6 weeks ago /bin/sh -c #(nop) MAINTAINER Phusion <[email protected]ā¦ 0B
<missing> 6 weeks ago /bin/sh -c #(nop) CMD ["/bin/bash"] 0B
<missing> 6 weeks ago /bin/sh -c mkdir -p /run/systemd && echo 'doā¦ 7B
<missing> 6 weeks ago /bin/sh -c sed -i 's/^#\s*\(deb.*universe\)$ā¦ 2.76kB
<missing> 6 weeks ago /bin/sh -c rm -rf /var/lib/apt/lists/* 0B
<missing> 6 weeks ago /bin/sh -c set -xe && echo '#!/bin/sh' > /ā¦ 745B
<missing> 6 weeks ago /bin/sh -c #(nop) ADD file:a3344b835ea6fdc56ā¦ 112MB
My changes to the Dockerfile WIP
IMAGE CREATED CREATED BY SIZE COMMENT
c2d1bb8dc4dc 3 minutes ago /bin/sh -c #(nop) ENTRYPOINT ["python" "/coā¦ 0B
0d35e01de91a 3 minutes ago /bin/sh -c apt-get update && apt-get insā¦ 627MB
1e1467300432 25 minutes ago /bin/sh -c #(nop) COPY file:d2dd9312ac40f85cā¦ 6B
ea975646156d 25 minutes ago /bin/sh -c #(nop) COPY file:7c13e4e876f9f1e8ā¦ 2.31kB
729cfdfc1133 25 minutes ago /bin/sh -c #(nop) COPY file:de9972b7e00aab74ā¦ 3.91kB
2fe519a788ee 25 minutes ago /bin/sh -c #(nop) COPY dir:0096577dd9215d190ā¦ 18.5kB
2e0d942637ba 25 minutes ago /bin/sh -c #(nop) COPY dir:d612083a96627f68dā¦ 100kB
b0eff52163b5 25 minutes ago /bin/sh -c #(nop) COPY file:48c2781c0c3cef57ā¦ 292B
614c90583c89 25 minutes ago /bin/sh -c #(nop) COPY file:91a496b3597da96aā¦ 1.57kB
bcdb76b0a87b 25 minutes ago /bin/sh -c #(nop) COPY dir:68c81018f4833e263ā¦ 453kB
b875347ab866 25 minutes ago /bin/sh -c #(nop) COPY dir:e0e90f8904a9ed21aā¦ 74.8kB
2efbadd7bd63 25 minutes ago /bin/sh -c #(nop) COPY dir:779e102780c9e2f15ā¦ 272kB
fee9b98779e3 25 minutes ago /bin/sh -c #(nop) COPY file:0ee2b2f83c2c5f46ā¦ 4.91kB
028ee1152551 25 minutes ago /bin/sh -c #(nop) COPY file:449f3516ab95493fā¦ 547B
35efeee52f01 25 minutes ago /bin/sh -c #(nop) COPY file:2475dfdfd7a6111cā¦ 654B
2270283cdc8f 25 minutes ago /bin/sh -c #(nop) WORKDIR /code 0B
b0f841a6cba8 25 minutes ago /bin/sh -c #(nop) ENV PYTHONPATH=:/code/Supā¦ 0B
d8e212be8c7e 25 minutes ago /bin/sh -c add-apt-repository -y ppa:ubuntugā¦ 1.82GB
d0b382d3989c 5 days ago /bin/sh -c #(nop) ENV DEBIAN_FRONTEND=noninā¦ 0B
166cfc3f6974 6 weeks ago /bin/sh -c #(nop) CMD ["/sbin/my_init"] 0B
<missing> 6 weeks ago /bin/sh -c #(nop) ENV DEBIAN_FRONTEND=teletyā¦ 0B
<missing> 6 weeks ago /bin/sh -c /bd_build/prepare.sh && /bd_builā¦ 97.3MB
<missing> 6 weeks ago /bin/sh -c #(nop) COPY dir:e7a5eda59d878c69cā¦ 39.4kB
<missing> 6 weeks ago /bin/sh -c #(nop) MAINTAINER Phusion <[email protected]ā¦ 0B
<missing> 6 weeks ago /bin/sh -c #(nop) CMD ["/bin/bash"] 0B
<missing> 6 weeks ago /bin/sh -c mkdir -p /run/systemd && echo 'doā¦ 7B
<missing> 6 weeks ago /bin/sh -c sed -i 's/^#\s*\(deb.*universe\)$ā¦ 2.76kB
<missing> 6 weeks ago /bin/sh -c rm -rf /var/lib/apt/lists/* 0B
<missing> 6 weeks ago /bin/sh -c set -xe && echo '#!/bin/sh' > /ā¦ 745B
<missing> 6 weeks ago /bin/sh -c #(nop) ADD file:a3344b835ea6fdc56ā¦ 112MB
The 1.82 GB of dependencies (APT & pip) is quite large as well as the SuperBuild at 627 MB. I cleaned up a bunch of redundant dependencies (listed more than once on various lines) and moved cleanup of APT and temp files to the RUN commands that create them. Iām assuming there is quite a bit more that I can clean up but I just donāt know the project well enough.
Have you guys thought about breaking ODM into a bunch of smaller projects and/or containers? Make each container specific to its process. For instance, OpenSFM should have its own container and use queues, pub/sub or a custom http api to communicate. You are already using docker so having people use compose to build and start a collection of containers is not asking much of the end user. It would also help make the project more (horizontally) scaleable while making it MUCH easier to reduce dependencies down to bare minimum. It is also a bit unnerving that pip dependencies are not version specific and you are not tagging docker builds with a version.
Update: I can open PR if you wanāt
From @smathermather on Fri Mar 16 2018 16:42:17 GMT+0000 (UTC)
@sberryman ā pull requests always welcome!
From @sberryman on Fri Mar 16 2018 16:45:55 GMT+0000 (UTC)
@smathermather Iām not sure it made a difference on download size, I know I at least cleaned up redundancies.
Iām running a job on the docker image after all the changes to ensure it still works. Do you guys have a pre-defined test(s) you use to verify the results of changes to the Dockerfile or underlying dependencies?
From @smathermather on Fri Mar 16 2018 16:50:21 GMT+0000 (UTC)
Our tests for ODM itself (as opposed to node-OpenDroneMap or WebODM) are embarrassingly lacking, but usually Dakota or Piero run pull requests through the paces manually.
Here is the rest of the conversation that did not get copied over: https://github.com/OpenDroneMap/OpenDroneMap/issues/781
1 Like
Thanks, the migration tool was pretty buggy.