Dockerfile Feedback


#1

From @sberryman on Fri Mar 09 2018 20:20:15 GMT+0000 (UTC)

FYI ONLY!

I noticed a recent commit to the Dockerfile and was quite interested. Then I started digging in to see what the Dockerfile was actually doing.

There are some glaring issues with the below section based on docker caching of layers. When reviewing the best practices for docker images, They state Always combine RUN apt-get update with apt-get install in the same RUN statement. The reason is that any apt-get install on subsequent layers are using the cached apt-get update from previous layers. Meaning if you re-run docker build on a machine that has already built ODM and don’t make any changes to the layers prior to apt-get update you’ll be using old cached lists and installing (potentially) old versions.

Then there is the issue of cleanup of APT. Running cleanup on a separate layer doesn’t remove the intermediate layers which have the cached data. So you either need to remove the cache lists in the same RUN command as you are doing apt-get update and apt-get install or just remove it from the dockerfile as it is useless.
https://github.com/OpenDroneMap/OpenDroneMap/blob/master/Dockerfile#L58-L59

Copied from original issue: https://github.com/OpenDroneMap/OpenDroneMap/issues/781


#2

From @dakotabenjamin on Fri Mar 09 2018 20:33:00 GMT+0000 (UTC)

This used to be the case, I’m not sure why it changed.


#3

From @sberryman on Fri Mar 09 2018 20:38:06 GMT+0000 (UTC)

Sorry, I’m not sure what you mean by “this used to be the case”. Are you talking about the removal of APT cache files? I’m pretty sure you only used to be able to get rid of intermediate layers by flattening the image, not very many people do that though.

Update: technically docker calls it -squash so hopefully I didn’t confuse you with ‘flatten’


#4

From @sberryman on Fri Mar 09 2018 21:33:59 GMT+0000 (UTC)

Any idea why libvtk6-dev header files are getting installed on line 56? They are already included in the massive dependency install phase on line 15.


#5

From @sberryman on Fri Mar 09 2018 21:52:03 GMT+0000 (UTC)

Want me to take a stab at rewriting it? How condensed do you want the final image? We can run almost everything in a single layer or I can try and split it out a bit. It would help to know what you tend to update. Does the build time of the image matter?

I could install the majority of deps near the top and then install build-essential, cmake, perform build and clean up files on another layer.

This would leave you with apt dependencies in the 1st layer, pip dependencies in 2nd, env & copy on a few more layers and finally build and cleanup on the final layer


#6

From @sberryman on Thu Mar 15 2018 14:49:01 GMT+0000 (UTC)

I played around with the Dockerfile a bit, here is docker history for a local build of ODM (96be8c4)

IMAGE               CREATED                  CREATED BY                                      SIZE                COMMENT
f4e1f13d0075        Less than a second ago   /bin/sh -c #(nop)  ENTRYPOINT ["python" "/co…   0B                  
69e075ebbf62        Less than a second ago   /bin/sh -c rm -rf /code/SuperBuild/download …   0B                  
8712d7541ebc        Less than a second ago   /bin/sh -c apt-get clean && rm -rf /var/lib/…   0B                  
64bb9330591e        Less than a second ago   /bin/sh -c apt-get install -y libvtk6-dev       299MB               
d6fe276c3611        Less than a second ago   /bin/sh -c apt-get -y remove libgl1-mesa-dri…   3.23MB              
be27339e4c81        Less than a second ago   /bin/sh -c cd SuperBuild && mkdir build && c…   2.46GB              
8695805db5a7        28 minutes ago           /bin/sh -c #(nop) COPY file:d2dd9312ac40f85c…   6B                  
fd3d44ede059        28 minutes ago           /bin/sh -c #(nop) COPY file:7c13e4e876f9f1e8…   2.31kB              
478225373e32        28 minutes ago           /bin/sh -c #(nop) COPY file:de9972b7e00aab74…   3.91kB              
77e3f134b1bd        28 minutes ago           /bin/sh -c #(nop) COPY dir:0096577dd9215d190…   18.5kB              
4a99d26ffff9        28 minutes ago           /bin/sh -c #(nop) COPY dir:d612083a96627f68d…   100kB               
8ffc941bc51f        28 minutes ago           /bin/sh -c #(nop) COPY file:48c2781c0c3cef57…   292B                
494c9c556426        28 minutes ago           /bin/sh -c #(nop) COPY file:91a496b3597da96a…   1.57kB              
fc361d40a3e0        28 minutes ago           /bin/sh -c #(nop) COPY dir:68c81018f4833e263…   453kB               
38efbc40404a        28 minutes ago           /bin/sh -c #(nop) COPY dir:e0e90f8904a9ed21a…   74.8kB              
91c2051df250        28 minutes ago           /bin/sh -c #(nop) COPY dir:779e102780c9e2f15…   272kB               
a66b4bf113bf        28 minutes ago           /bin/sh -c #(nop) COPY file:0ee2b2f83c2c5f46…   4.91kB              
fe3fc6309d03        28 minutes ago           /bin/sh -c #(nop) COPY file:449f3516ab95493f…   547B                
cb47c53f7025        28 minutes ago           /bin/sh -c #(nop) COPY file:2475dfdfd7a6111c…   654B                
a3d14dbfe747        28 minutes ago           /bin/sh -c #(nop) WORKDIR /code                 0B                  
5035510c7737        28 minutes ago           /bin/sh -c mkdir /code                          0B                  
9858aeeed408        28 minutes ago           /bin/sh -c #(nop)  ENV LD_LIBRARY_PATH=:/cod…   0B                  
53a7d71dc731        28 minutes ago           /bin/sh -c #(nop)  ENV PYTHONPATH=:/code/Sup…   0B                  
c8ec01695612        28 minutes ago           /bin/sh -c #(nop)  ENV PYTHONPATH=:/code/Sup…   0B                  
2c91acaea5aa        28 minutes ago           /bin/sh -c pip install -U PyYAML exifread gp…   353MB               
5a7e8746e9ae        30 minutes ago           /bin/sh -c pip install setuptools               2.85MB              
96b6e0695ffb        30 minutes ago           /bin/sh -c pip install --upgrade pip            8.22MB              
40b6d629abeb        30 minutes ago           /bin/sh -c apt-get remove libdc1394-22-dev      0B                  
91f8de57c68f        30 minutes ago           /bin/sh -c apt-get install --no-install-reco…   1.67GB              
6fd16cdd33fe        36 minutes ago           /bin/sh -c apt-get update -y                    686kB               
d4d0361edc78        36 minutes ago           /bin/sh -c add-apt-repository -y ppa:george-…   6.27kB              
076942099b74        36 minutes ago           /bin/sh -c add-apt-repository -y ppa:ubuntug…   15.9kB              
f6ae1c8f8633        36 minutes ago           /bin/sh -c apt-get install software-properti…   0B                  
63771a799817        36 minutes ago           /bin/sh -c apt-get update -y                    43.1MB              
d0b382d3989c        5 days ago               /bin/sh -c #(nop)  ENV DEBIAN_FRONTEND=nonin…   0B                  
166cfc3f6974        6 weeks ago              /bin/sh -c #(nop) CMD ["/sbin/my_init"]         0B                  
<missing>           6 weeks ago              /bin/sh -c #(nop) ENV DEBIAN_FRONTEND=telety…   0B                  
<missing>           6 weeks ago              /bin/sh -c /bd_build/prepare.sh &&  /bd_buil…   97.3MB              
<missing>           6 weeks ago              /bin/sh -c #(nop) COPY dir:e7a5eda59d878c69c…   39.4kB              
<missing>           6 weeks ago              /bin/sh -c #(nop) MAINTAINER Phusion <info@p…   0B                  
<missing>           6 weeks ago              /bin/sh -c #(nop)  CMD ["/bin/bash"]            0B                  
<missing>           6 weeks ago              /bin/sh -c mkdir -p /run/systemd && echo 'do…   7B                  
<missing>           6 weeks ago              /bin/sh -c sed -i 's/^#\s*\(deb.*universe\)$…   2.76kB              
<missing>           6 weeks ago              /bin/sh -c rm -rf /var/lib/apt/lists/*          0B                  
<missing>           6 weeks ago              /bin/sh -c set -xe   && echo '#!/bin/sh' > /…   745B                
<missing>           6 weeks ago              /bin/sh -c #(nop) ADD file:a3344b835ea6fdc56…   112MB               

My changes to the Dockerfile WIP

IMAGE               CREATED             CREATED BY                                      SIZE                COMMENT
c2d1bb8dc4dc        3 minutes ago       /bin/sh -c #(nop)  ENTRYPOINT ["python" "/co…   0B                  
0d35e01de91a        3 minutes ago       /bin/sh -c apt-get update &&     apt-get ins…   627MB               
1e1467300432        25 minutes ago      /bin/sh -c #(nop) COPY file:d2dd9312ac40f85c…   6B                  
ea975646156d        25 minutes ago      /bin/sh -c #(nop) COPY file:7c13e4e876f9f1e8…   2.31kB              
729cfdfc1133        25 minutes ago      /bin/sh -c #(nop) COPY file:de9972b7e00aab74…   3.91kB              
2fe519a788ee        25 minutes ago      /bin/sh -c #(nop) COPY dir:0096577dd9215d190…   18.5kB              
2e0d942637ba        25 minutes ago      /bin/sh -c #(nop) COPY dir:d612083a96627f68d…   100kB               
b0eff52163b5        25 minutes ago      /bin/sh -c #(nop) COPY file:48c2781c0c3cef57…   292B                
614c90583c89        25 minutes ago      /bin/sh -c #(nop) COPY file:91a496b3597da96a…   1.57kB              
bcdb76b0a87b        25 minutes ago      /bin/sh -c #(nop) COPY dir:68c81018f4833e263…   453kB               
b875347ab866        25 minutes ago      /bin/sh -c #(nop) COPY dir:e0e90f8904a9ed21a…   74.8kB              
2efbadd7bd63        25 minutes ago      /bin/sh -c #(nop) COPY dir:779e102780c9e2f15…   272kB               
fee9b98779e3        25 minutes ago      /bin/sh -c #(nop) COPY file:0ee2b2f83c2c5f46…   4.91kB              
028ee1152551        25 minutes ago      /bin/sh -c #(nop) COPY file:449f3516ab95493f…   547B                
35efeee52f01        25 minutes ago      /bin/sh -c #(nop) COPY file:2475dfdfd7a6111c…   654B                
2270283cdc8f        25 minutes ago      /bin/sh -c #(nop) WORKDIR /code                 0B                  
b0f841a6cba8        25 minutes ago      /bin/sh -c #(nop)  ENV PYTHONPATH=:/code/Sup…   0B                  
d8e212be8c7e        25 minutes ago      /bin/sh -c add-apt-repository -y ppa:ubuntug…   1.82GB              
d0b382d3989c        5 days ago          /bin/sh -c #(nop)  ENV DEBIAN_FRONTEND=nonin…   0B                  
166cfc3f6974        6 weeks ago         /bin/sh -c #(nop) CMD ["/sbin/my_init"]         0B                  
<missing>           6 weeks ago         /bin/sh -c #(nop) ENV DEBIAN_FRONTEND=telety…   0B                  
<missing>           6 weeks ago         /bin/sh -c /bd_build/prepare.sh &&  /bd_buil…   97.3MB              
<missing>           6 weeks ago         /bin/sh -c #(nop) COPY dir:e7a5eda59d878c69c…   39.4kB              
<missing>           6 weeks ago         /bin/sh -c #(nop) MAINTAINER Phusion <info@p…   0B                  
<missing>           6 weeks ago         /bin/sh -c #(nop)  CMD ["/bin/bash"]            0B                  
<missing>           6 weeks ago         /bin/sh -c mkdir -p /run/systemd && echo 'do…   7B                  
<missing>           6 weeks ago         /bin/sh -c sed -i 's/^#\s*\(deb.*universe\)$…   2.76kB              
<missing>           6 weeks ago         /bin/sh -c rm -rf /var/lib/apt/lists/*          0B                  
<missing>           6 weeks ago         /bin/sh -c set -xe   && echo '#!/bin/sh' > /…   745B                
<missing>           6 weeks ago         /bin/sh -c #(nop) ADD file:a3344b835ea6fdc56…   112MB               

The 1.82 GB of dependencies (APT & pip) is quite large as well as the SuperBuild at 627 MB. I cleaned up a bunch of redundant dependencies (listed more than once on various lines) and moved cleanup of APT and temp files to the RUN commands that create them. I’m assuming there is quite a bit more that I can clean up but I just don’t know the project well enough.

Have you guys thought about breaking ODM into a bunch of smaller projects and/or containers? Make each container specific to its process. For instance, OpenSFM should have its own container and use queues, pub/sub or a custom http api to communicate. You are already using docker so having people use compose to build and start a collection of containers is not asking much of the end user. It would also help make the project more (horizontally) scaleable while making it MUCH easier to reduce dependencies down to bare minimum. It is also a bit unnerving that pip dependencies are not version specific and you are not tagging docker builds with a version.

Update: I can open PR if you wan’t


#7

From @smathermather on Fri Mar 16 2018 16:42:17 GMT+0000 (UTC)

@sberryman – pull requests always welcome!


#8

From @sberryman on Fri Mar 16 2018 16:45:55 GMT+0000 (UTC)

@smathermather I’m not sure it made a difference on download size, I know I at least cleaned up redundancies.

I’m running a job on the docker image after all the changes to ensure it still works. Do you guys have a pre-defined test(s) you use to verify the results of changes to the Dockerfile or underlying dependencies?


#9

From @smathermather on Fri Mar 16 2018 16:50:21 GMT+0000 (UTC)

Our tests for ODM itself (as opposed to node-OpenDroneMap or WebODM) are embarrassingly lacking, but usually Dakota or Piero run pull requests through the paces manually.


#10

Here is the rest of the conversation that did not get copied over: https://github.com/OpenDroneMap/OpenDroneMap/issues/781


#11

Thanks, the migration tool was pretty buggy.