Docker Notes
Posted on Sat 20 February 2021 in Docker
This post contains my notes about Docker, specifically about Learn Docker in a Month of Lunches. The videos are available in YouTube.
Reasons for using Docker
Docker makes heavy automation easier
You often see Docker in the context of DevOPs because it is super easy to automate. So this idea of a pipeline that takes your code, packages it, runs it in a container, runs a bunch of tests, if everything passes then it goes, and pushes it to live. It is simplified with Docker because it is so automation heavy. It is much easier to do automation in Docker than to manually create stuff.
Understanding Docker and running "Hello, World"
The image is the class.
The container is an instance of the class.
You can have multiple containers from the same image.
Hello, World and nothing more
Run the following command to create a container. It will print the host name and IP address of the container:
$ sudo docker container run diamol/ch02-hello-diamol
This is the output:
Unable to find image 'diamol/ch02-hello-diamol:latest' locally
latest: Pulling from diamol/ch02-hello-diamol
31603596830f: Pull complete
93931504196e: Pull complete
d7b1f3678981: Pull complete
Digest: sha256:c968d535a42e4298cc229fb3450c900d8917bbe0d2ab819980880ddd3678000d
Status: Downloaded newer image for diamol/ch02-hello-diamol:latest
---------------------
Hello from Chapter 2!
---------------------
My name is:
d27951b34539
---------------------
Im running on:
Linux 4.15.0-133-generic x86_64
---------------------
My address is:
inet addr:172.17.0.2 Bcast:172.17.255.255 Mask:255.255.0.0
---------------------
Consider these lines from that log:
Unable to find image 'diamol/ch02-hello-diamol:latest' locally
latest: Pulling from diamol/ch02-hello-diamol
Status: Downloaded newer image for diamol/ch02-hello-diamol:latest
Notice that we requested the diamol/ch02-hello-diamol image. However, Docker pulled (downloaded) an image named diamol/ch02-hello-diamol:latest. In fact, there are multiple tags for that image in Docker Hub. We will talk about that later.
Now, check the output of the application inside the container:
---------------------
Hello from Chapter 2!
---------------------
My name is:
69e8490d4103
---------------------
Im running on:
Linux 4.15.0-133-generic x86_64
---------------------
My address is:
inet addr:172.17.0.2 Bcast:172.17.255.255 Mask:255.255.0.0
---------------------
If you run that command again, the name and address will be different:
$ sudo docker container run diamol/ch02-hello-diamol
---------------------
Hello from Chapter 2!
---------------------
My name is:
986ce2ab3b38
---------------------
Im running on:
Linux 4.15.0-133-generic x86_64
---------------------
My address is:
inet addr:172.17.0.2 Bcast:172.17.255.255 Mask:255.255.0.0
---------------------
Here, name and address are the host name and IP address of the container, respectively. They are provided by Docker.
The container could package anything, like a script that runs an application that deploys a bunch of stuff in the cloud or runs a bunch of other tests or anything. And all I need to do is to run
docker container runand as long the application image is set up in a way that it is easy to use, it will just work in the same way on my machine, it will do everything that it's kind of scripted to do.
The container is basically a virtual environment, similar to those created with mkvirtualenv in Python:
What you get when you run your application container is a virtual environment that Docker creates for you. It is like a box with an application in it. Docker provides the disk, the IP address, the hostname, and the virtualized environment. It is like a virtual machine except that Docker is super lightweight because the process that runs the application runs directly on the machine, so there is no kind of operating system layer in between. So the container itself uses the operating system of the machine where you are running Docker. That's why containers are so fast and why it takes a very little overhead to run lots of containers on one machine.
If you are curious, Docker Hub hosts the image diamol/ch02-hello-diamol in:
https://hub.docker.com/r/diamol/ch02-hello-diamol
I visited that URL looking for a Dockerfile there. Instead, I found something like this:
ADD file:a0afd0b0db7f9ee9496186ead087ec00edd1386ea8c018557d15720053f7308e in /
CMD ["/bin/sh"]
COPY file:3c74b4e9eab9986d0a784601afaa462ea1c79a00da03803ff19a4e68c72bec78 in .
/bin/sh -c chmod +x cmd.sh
CMD ["/bin/sh" "-c" "./cmd.sh"]
What are those hash values? At first I was looking for the script that outputs the name and address of the container. From the snippet shown above, the container may have ran this command to generate such an output:
/bin/sh -c ./cmd.sh
where cmd.sh may contain commands like uname -n for printing the host name and something like ifconfig wlp2s0 for the IP address. However, cmd.sh is not available in Docker Hub. Instead, we got a hash value that refers to that shell script. According to this post:
That Docker Hub history view doesn't show the actual Dockerfile; instead, it shows content essentially extracted from the
docker historyof the image. That doesn't preserve the specific details you're looking for: it doesn't remember the names of base images, or the build-context file names of things that getADDed orCOPYed in.
In turns out that a user had the same question. Another user provide a possible Dockerfile for his question:
FROM scratch
ADD rootfs.tar.xz /
CMD ["bash"]
If you look at
docker historyor the Docker Hub history view you cite, you should be able to see these same steps happening. TheADD file:4b0... in /corresponds to theADD rootfs.tar.gz /, and the second line is theCMD ["bash"]. It is not split up by Dockerfile or image, and the original filenames fromADDaren't saved. (You couldn't reproduce the image anyways without the contents of therootfs.tar.gz, so it's merely slightly helpful to know its filename but not essential.)The
ADD file:hash in /pathsyntax is not standard Dockerfile syntax (the wordinin particular is not part of it). I'm not sure there's a reliable way to translate from the host file or URL to the hash, but building the image and looking at itsdocker historywould tell you (assuming you've got a perfect match for the file metadata). There's no way to get back to the original filename or syntax, and definitely no way to get back to the file contents.
In summary, we cannot access files added or copied into the container. We only will see references to those file names as hash values. Even if we knew the names, we would still need them for creating the image.
We can use the following command to get a list of the images in our system:
$ sudo docker images
This is the output:
REPOSITORY TAG IMAGE ID CREATED SIZE
diamol/ch02-hello-diamol latest 387f84126dbc 5 months ago 5.55MB
According to the previous post, we can use docker history <image_id> to inspect the image:
$ sudo docker history 387f84126dbc
IMAGE CREATED CREATED BY SIZE COMMENT
387f84126dbc 5 months ago /bin/sh -c #(nop) CMD ["/bin/sh" "-c" "./cm… 0B
<missing> 5 months ago /bin/sh -c chmod +x cmd.sh 294B
<missing> 5 months ago /bin/sh -c #(nop) COPY file:3c74b4e9eab9986d… 294B
<missing> 10 months ago /bin/sh -c #(nop) CMD ["/bin/sh"] 0B
<missing> 10 months ago /bin/sh -c #(nop) ADD file:a0afd0b0db7f9ee94… 5.55MB
It shows the same content given in Docker Hub but in reverse order.
After taking a break, I remembered the repository of diamol in GitHub. There you will find the Dockerfiles of the exercises. In this case, the Dockerfile is the following:
FROM alpine:3.9
COPY cmd.sh .
RUN chmod +x cmd.sh
CMD ./cmd.sh
and cmd.sh:
echo ---------------------
echo Hello from Chapter 2!
echo ---------------------
echo My name is:
echo $(hostname)
echo ---------------------
echo I''m running on:
echo $(uname -s -r -m)
echo ---------------------
echo My address is:
echo $(ifconfig eth0 | grep inet)
echo ---------------------
Interactive mode
Run the following command to create a container and start a shell on it:
$ sudo docker container run --interactive --tty diamol/base
That single command creates a new container and starts a remote connection similar to SSH. This is the output:
Unable to find image 'diamol/base:latest' locally
latest: Pulling from diamol/base
31603596830f: Already exists
Digest: sha256:8a0a4e35241af4f4b16138850c88e4b7328f4a36eee0769e58e13812c7d02adc
Status: Downloaded newer image for diamol/base:latest
/ #
Here, Docker downloaded an image called diamol/base:latest.
Notice the / # at the end of the log. We are connected to the container as it was a remote server. Before going any further, run the following command to verify that you have two images in your system:
$ sudo docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
diamol/ch02-hello-diamol latest 387f84126dbc 5 months ago 5.55MB
diamol/base latest 78a2ce922f86 10 months ago 5.55MB
and the following command to show the containers:
$ sudo docker container ls --all
CONTAINER ID IMAGE COMMAND CREATED STATUS
f62620ff326d diamol/base "/bin/sh" 7 minutes ago Up 7 minutes
d27951b34539 diamol/ch02-hello-diamol "/bin/sh -c ./cmd.sh" 29 minutes ago Exited (0) 29 minutes ago
Now, we have two images:
diamol/ch02-hello-diamoldiamol/base
and two containers:
f62620ff326d(running/bin/sh)d27951b34539(not running anymore)
Back to our interactive mode. We are connected to the container. It is like connecting to a remote machine. Let's type a few commands:
/ # hostname
f62620ff326d
/ # date
Sun Feb 21 04:51:48 UTC 2021
Notice that the hostname is the same as the CONTAINER ID of the container shown above. The hostname is the name that the container sees from inside and that's part of the virtual environment that Docker constructs. On the other hand, the date is shared from the host operating system.
Running a web server in a container
Most of the time you create a container for running a service or an application until you stop (or replace) the container. The next example shows you how to run a really simple website in a container.
The containers are only running as long as the application in the container is running. [...] the container is just a wrapper around an application process. If the process ends, the containers is gone with it. It still exists, you can still get to the disk and you can start it up if you want to but [if it is inactive] it is not using any computer power.
That's why our first container finished after printing its hostname and address. Since the application cmd.sh finished, the container became inactive.
The thing a lot of beginners don't quite understand when they start running their own applications is they build their application package to run in a container. When they run the container it starts and finishes and they don't know why it isn't still running. It's because they haven't configured it to start their process and make sure that process keeps on running.
Run the following command:
$ sudo docker container run --detach --publish 8088:80 diamol/ch02-hello-diamol-web
Here, deatch means start that container, put it in the background, keep it running. As long as the process is running the container keeps running. The publish flag tells Docker to allow network traffic into this container from the host (network traffic is disabled by default). In this case, my machine is listening the port 8088 and the container is listening the port 80. When traffic comes in on the port 8088 in my computer, it is sent to the container on the port 80. This is my output:
Unable to find image 'diamol/ch02-hello-diamol-web:latest' locally
latest: Pulling from diamol/ch02-hello-diamol-web
aad63a933944: Pull complete
29ade582b51e: Pull complete
7e41ad5b6f9c: Pull complete
ebf61b47b4ca: Pull complete
9c060bce4eae: Pull complete
accccc30537e: Pull complete
Digest: sha256:6f1e3ddd38362b633fd200f6182b5a9d83bb17b602a0d61f12676efe87c58d65
Status: Downloaded newer image for diamol/ch02-hello-diamol-web:latest
c134006c1eb857e4031955a2aad220a33b84f456cd80dde8b941f589973d2e82
Unlike the two previous examples, the last line of the log shows the ID of the container:
c134006c1eb857e4031955a2aad220a33b84f456cd80dde8b941f589973d2e82
You can confirm this as follows:
$ sudo docker container ls
CONTAINER ID IMAGE COMMAND STATUS PORTS
c134006c1eb8 diamol/ch02-hello-diamol-web "httpd-foreground" Up 7 minutes 0.0.0.0:8088->80/tcp
f62620ff326d diamol/base "/bin/sh" Up 2 hours
Now, you can go to http://localhost:8088 in your machine and see a simple HTML page.
Note I tested whether the container was available after restarting my computer. It was not. However, chances are you could still be able to see the page in the browser. That is because the page is in the cache of the browser. Clear the cache and try again. The page will not be available the next time.
We can also check the work load in the container from the memory and CPU point of view as follows:
$ sudo docker container stats c13
Finally, we will remove the containers. These are our containers so far:
$ sudo docker container ls --all
CONTAINER ID IMAGE COMMAND NAMES
c134006c1eb8 diamol/ch02-hello-diamol-web "httpd-foreground" wizardly_bartik
f62620ff326d diamol/base "/bin/sh" boring_payne
d27951b34539 diamol/ch02-hello-diamol "/bin/sh -c ./cmd.sh" loving_liskov
If the container is still running, we need the --force flag to force removal. We can use the following command to remove a container:
$ sudo docker container rm <container_name>
For example:
$ sudo docker container rm d279
will remove the d27951b34539 container.
Optionally, we can use the next command to remove all your containers:
$ sudo docker container rm --force $(docker container ls --all --quiet)
This is the output of the nested command docker container ls --all --quiet, in case you are wondering what it does:
$ docker container ls --all --quiet
c134006c1eb8
f62620ff326d
Thus, the previous docker container rm command will remove all the containers. Let's do it:
$ sudo docker container rm --force $(docker container ls --all --quiet)
c134006c1eb8
f62620ff326d
The output of rm is the name of the container. Since we remove two containers, there are two lines.
Connecting to the container
I wondered whether I could connect to the container with Apache shown before. I used these commands:
$ sudo docker container run --detach --publish 8088:80 diamol/ch02-hello-diamol-web
99f1db4337a1ee84f5090af2965ff720e51fe66c4acffd5663d5c12dc603a7d3
$ sudo docker container stop 99f # stop the container
$ sudo docker container start --attach --interactive 99f # start the container interactively
I could also use this single command (same as above but without the --detach flag):
$ sudo docker container run --publish 8088:80 diamol/ch02-hello-diamol-web
Note I also used these commands:
$ sudo docker container run --interactive --tty --publish 8088:80 diamol/ch02-hello-diamol-web
That command finished the container after creating it (I do not why). It showed this log:
[Mon Feb 22 06:12:01.697126 2021] [mpm_event:notice] [pid 1:tid 140256642780488] AH00489: Apache/2.4.41 (Unix) configured -- resuming normal operations
[Mon Feb 22 06:12:01.697310 2021] [core:notice] [pid 1:tid 140256642780488] AH00094: Command line: 'httpd -D FOREGROUND'
[Mon Feb 22 06:12:01.727214 2021] [mpm_event:notice] [pid 1:tid 140256642780488] AH00492: caught SIGWINCH, shutting down gracefully
Then, I tried this command:
$ sudo docker container run --interactive --publish 8088:80 diamol/ch02-hello-diamol-web
That container worked.
In any case, we will see this output:
[Mon Feb 22 06:13:51.953738 2021] [mpm_event:notice] [pid 1:tid 140650727468360] AH00489: Apache/2.4.41 (Unix) configured -- resuming normal operations
[Mon Feb 22 06:13:51.953788 2021] [core:notice] [pid 1:tid 140650727468360] AH00094: Command line: 'httpd -D FOREGROUND'
172.17.0.1 - - [22/Feb/2021:06:13:56 +0000] "GET / HTTP/1.1" 200 260
172.17.0.1 - - [22/Feb/2021:06:13:56 +0000] "GET /favicon.ico HTTP/1.1" 404 196
172.17.0.1 - - [22/Feb/2021:06:13:58 +0000] "GET / HTTP/1.1" 304 -
172.17.0.1 - - [22/Feb/2021:06:13:59 +0000] "GET / HTTP/1.1" 304 -
and a cursor blinking at the end of the shell. It means we are connected to the container but we cannot run commands in the shell. We can only see the output messages of the web server. In other words, just because we can make a remote connection to the container it does not mean that we can run commands on it. This is because the container is constructed with this Dockerfile
FROM diamol/apache
COPY html/ /usr/local/apache2/htdocs/
The diamol/apache is available in Docker Hub. As we discuss before, its Dockerfile is partially hashed. Although, the last command shows us that there is a service running in the background:
CMD ["httpd-foreground"]
Now, let's talk about the diamol/base image. If you remember, we employed that image for an interactive session: we created a container and then we connected our terminal using this command:
$ sudo docker container run --interactive --tty diamol/base
That image is available in DockerHub as well. The last line of that Dockerfile is:
CMD ["/bin/sh"]
Thus, I think the reason we could not connect to the Apache container is because it was not configured with a shell running in the background. Unlike the Apache container, the interactive container does have a shell running in the background. That why we can run commands after creating a remote connection to the container.
One more note about these two commands:
$ sudo docker container start --attach --interactive 99f # start the container interactively
$ sudo docker container run --interactive --publish 8088:80 diamol/ch02-hello-diamol-web
Since I am a beginner I did not understand the difference between both commands at a first glance. However, I think the first command allows us to create an interactive session with an existing container, whereas the second command creates an interactive session with a new container.
Version of Docker
You can use the following command to get the version of the client and server of Docker:
$ docker version
This is my output:
Client: Docker Engine - Community
Version: 19.03.1
...
Server: Docker Engine - Community
Engine:
Version: 19.03.1
...
The Docker server happens to be in my computer but it could be in a remote server. In any case, the client (the command line interface) communicates with the server via REST API.
One last thing
The tutorial ends with a lab. Here, we are asked to perform a given task by ourselves. In this case, it is as follows:
Your task is to run the website container and replace the
index.htmlfile so when you browse to the container you see a different homepage. Remember that the container has its own filesystem, and in this application, the website is serving files that are on the container's filesystem.
There are a few hints:
- You can run
docker containerto get a list of all the actions you can perform on a container.- Add
--helpto anydockercommand, and you'll see more detailed help text.- In the
diamol/ch02-hello-diamol-webDocker image, the content from the website is served from the directory/usr/local/apache2/htdocs.
Let's begin creating a new container:
sudo docker container run --detach --publish 8088:80 diamol/ch02-hello-diamol-web
c23c504ca92be76f3e8b43f34372998b321e355e8c812e73854dd607db46009f
You can go to http://localhost:8088 to verify the container is running. Now, we need to create a new HTML page and copy it into the container. I will use this minimal HTML base:
<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<meta name="description" content="">
<title>Minimal base.html</title>
</head>
<body>
<!-- Delete this part -->
<h1>base.html</h1>
<p>The absolute minimum <code>base.html</code> to get your project started.</p>
<h3>Usage:</h3>
<pre>
curl https://basehtml.xyz > base.html
</pre>
<a href="https://github.com/sesh/base.html">more info</a>
</body>
</html>
Save that file as index.html in your $HOME.
Now, we need a way to copy it to the container. We can use docker container cp:
$ sudo docker container cp --help
Usage: docker container cp [OPTIONS] CONTAINER:SRC_PATH DEST_PATH|-
docker cp [OPTIONS] SRC_PATH|- CONTAINER:DEST_PATH
Copy files/folders between a container and the local filesystem
According to this post, we can use it as follows:
Copy a specific file TO the container
$ docker cp foo.txt mycontainer:/foo.txt
Copy a specific file FROM the container
$ docker cp mycontainer:/foo.txt foo.txt
Given this, we can use the following command to copy our index.html file in the container:
$ sudo docker container cp index.html c23c:/usr/local/apache2/htdocs
where c23c is a part of the name of the container. Clear the cache of your browser and then go to http://localhost:8088. You will see our new index file.
The author gives a solution in his repository. He also provides a few notes regarding using Docker on Windows. In that case, you first need to stop the container before copying the new index file. Then, you can start the container again. The whole process is given below:
$ sudo docker container stop c23c
$ sudo docker container cp index.html c23c:/usr/local/apache2/htdocs
$ sudo docker container start c23c
Commands
List all active containers
$ sudo docker container ls
CONTAINER ID IMAGE COMMAND CREATED ...
f62620ff326d diamol/base "/bin/sh" 51 minutes ago ...
Remember, the CONTAINER ID is the value given by thehostname command within the container.
List all containers This shows all the containers (active and inactive containers).
$ sudo docker container ls --all
CONTAINER ID IMAGE COMMAND CREATED STATUS
f62620ff326d diamol/base "/bin/sh" 53 minutes ago Up 53
d27951b34539 diamol/ch02-hello-diamol "/bin/sh -c ./cmd.sh" About an hour ago Exited (0)
Here, the container d27951b34539 is inactive since it ran a script and finished. This means that the container is not using any more compute resources like CPU or RAM. Although the container is inactive, its environment still exists so its virtual disk still exists.
Stop and start a container
$ sudo docker container stop <container_name>
$ sudo docker container start <container_name>
$ sudo docker container stop c23c
$ sudo docker container start c23c
Copy files to/from the container You do not need sudo.
$ docker container cp <container_name:src_path> <dest_path>
$ docker container cp <src_path> <container_name:dest_path>
From the host to the container:
$ docker container cp index.html c23c:/usr/local/apache2/htdocs
From the container to the host:
$ docker container cp c23c:/usr/local/apache2/htdocs/index.html new_index.html
If you want to copy a directory from the host to the container, use /. at the end of <src_path>. For example:
$ docker container cp test_dir/. c23c:/usr/local/apache2/htdocs
You can verify it as follows:
$ docker container exec c23c ls /usr/local/apache2/htdocs
index.html
sample
sample1
sample2
If <src_path> is a directory and does not end with /., it will copy the whole directory:
$ docker container cp test_dir c23c:/usr/local/apache2/htdocs
Check it:
docker container exec c23c ls /usr/local/apache2/htdocs/test_dir
index.html
sample
sample1
sample2
Show files in a container You do not need sudo.
$ docker container exec <container_name> ls <path>
$ docker container exec c23c ls /usr/local/apache2/htdocs
Show processes running in a container
$ sudo docker container top <container_name>
$ sudo docker container top f62
UID PID PPID C STIME TTY TIME CMD
root 11334 11309 0 22:05 pts/0 00:00:00 /bin/sh
It shows the processes that are running inside the container f62620ff326d.
Show the log of a container
$ sudo docker container logs <container_name>
$ sudo docker container logs f62
/ # hostname
f62620ff326d
/ # date
Sun Feb 21 04:51:48 UTC 2021
It will show the output of the container generated by a batch process, a web service, or by our interactive session.
Show the work load of a container
$ sudo docker container stats <container_name>
$ sudo docker container stats c13
Inspect a container
$ sudo docker container inspect <container_name>
$ sudo docker container inspect f62
[
{
"Id": "f62620ff326dc69fe12f82ef0d897effe43747be7613694e5326c4e21996758f",
"Created": "2021-02-21T04:05:43.148777296Z",
"Path": "/bin/sh",
"Args": [],
"State": {
"Status": "running",
"Running": true,
"Paused": false,
"Restarting": false,
"OOMKilled": false,
"Dead": false,
"Pid": 11334,
"ExitCode": 0,
"Error": "",
"StartedAt": "2021-02-21T04:05:43.724317635Z",
"FinishedAt": "0001-01-01T00:00:00Z"
},
...
"LogPath": "/var/lib/docker/containers/f626../f626...-json.log",
...
"Config": {
"Hostname": "f62620ff326d",
"Domainname": "",
"User": "",
"AttachStdin": true,
"AttachStdout": true,
"AttachStderr": true,
"Tty": true,
"OpenStdin": true,
"StdinOnce": true,
"Env": [
"PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
],
"Cmd": [
"/bin/sh"
],
"Image": "diamol/base",
"Volumes": null,
"WorkingDir": "",
"Entrypoint": null,
"OnBuild": null,
"Labels": {}
},
...
}
]
Remove a container You can use the --force to remove an active container:
$ sudo docker container rm <container_name>
$ sudo docker container rm d279
Remove all containers You can use the --force to remove an active container:
$ sudo docker container rm --force $(docker container ls --all --quiet)