Docker notes

This is my first play with Docker in anger. This tutorial (by Emanuele Cesena) helped me get Beam up and running with Docker very quickly. The Beam part is a bit too old now but it gives a good introduction to both technologies.


Instances

With Docker Compose, you can have a cluster of Docker containers. In a directory with a suitable docker-compose.yml file, run docker-compose (which "define[s] and run[s] multi-container applications with Docker" according to the man pages):

$ docker-compose up -d
...
190fcbe8a871: Pull complete
22b3697a4a0a: Pull complete
Digest: sha256:3d993a92474808f4920ccd346c0fad3f43b7c5d36539c723ac7947c406974300
Status: Downloaded newer image for dataradiant/beam-flink:latest
Creating dockerbeamflink_jobmanager_1
Creating dockerbeamflink_taskmanager_1
$ docker ps
CONTAINER ID        IMAGE                    COMMAND                  CREATED             STATUS              PORTS                                                                             NAMES
87ec0891bc7b        dataradiant/beam-flink   "/usr/local/flink/bi…"   2 hours ago         Up 2 hours          6121-6123/tcp, 0.0.0.0:32768->22/tcp                                              dockerbeamflink_taskmanager_1

a057b8d8ff64        dataradiant/beam-flink   "/usr/local/flink/bi…"   2 hours ago         Up 2 hours          6123/tcp, 0.0.0.0:220->22/tcp, 0.0.0.0:48080->8080/tcp, 0.0.0.0:48081->8081/tcp   dockerbeamflink_jobmanager_1


Filesystems

$ docker run -t -i dataradiant/beam-flink  /bin/bash
root@54592dfb8040:/# ls -ltr
total 64

drwxr-xr-x   2 root root 4096 Apr 10  2014 mnt
...

Hey! That's not my root directory (StackOverflow)!

Note, this is not the same as logging onto a running container. 

root@54592dfb8040:/# /usr/java/default/bin/jps 

71 Jps

To do that, run this:

$ docker exec -it dockerbeamflink_taskmanager_1 /bin/bash

root@87ec0891bc7b:/# /usr/java/default/bin/jps 

238 TaskManager

323 Jps

Where are the images stored? "By default this will be aufs but can fall back to overlay, overlay2, btrfs, devicemapper or zfs depending on your kernel support."

https://stackoverflow.com/questions/19234831/where-are-docker-images-stored-on-the-host-machine

On my Ubuntu 16.04:

$ sudo du -sh /var/lib/docker/overlay2

11G /var/lib/docker/overlay2

"The main mechanics of OverlayFS relate to the merging of directory access when both filesystems present a directory for the same name." (Wikipedia)

Networking

The Docker daemon appears to act like a software router. That is, I access a URL on http://172.17.0.1:48080 but neither of my running Docker images have this IP address:

root@87ec0891bc7b:/# ifconfig 
eth0      Link encap:Ethernet  HWaddr 02:42:ac:11:00:03  
          inet addr:172.17.0.3  Bcast:172.17.255.255  Mask:255.255.0.0

root@a057b8d8ff64:/# ifconfig 

eth0      Link encap:Ethernet  HWaddr 02:42:ac:11:00:02  

          inet addr:172.17.0.2  Bcast:172.17.255.255  Mask:255.255.0.0

Instead, it is the host OS that has it:

henryp@corsair:~$ ifconfig 
...
docker0   Link encap:Ethernet  HWaddr 02:42:02:bc:6e:af  
          inet addr:172.17.0.1  Bcast:172.17.255.255  Mask:255.255.0.0
...

henryp@corsair:~$ sudo netstat -nap | grep 48080

tcp        0      0 172.17.0.1:53756        172.17.0.1:48080        ESTABLISHED 5084/chrome --type=

tcp6       0      0 :::48080                :::*                    LISTEN      17660/docker-proxy

and the Docker daemon does some NATing because the instances don't have that port open:

root@a057b8d8ff64:/# netstat -nap | grep 48080

root@a057b8d8ff64:/#

root@87ec0891bc7b:/# netstat -nap | grep 48080

root@87ec0891bc7b:/#

Housekeeping

Stopping a Docker instance is simple: 

henryp@corsair:~$ docker ps

CONTAINER ID        IMAGE                    COMMAND                  CREATED             STATUS              PORTS                                                                             NAMES

87ec0891bc7b        dataradiant/beam-flink   "/usr/local/flink/bi…"   20 hours ago        Up 20 hours         6121-6123/tcp, 0.0.0.0:32768->22/tcp                                              dockerbeamflink_taskmanager_1

a057b8d8ff64        dataradiant/beam-flink   "/usr/local/flink/bi…"   20 hours ago        Up 20 hours         6123/tcp, 0.0.0.0:220->22/tcp, 0.0.0.0:48080->8080/tcp, 0.0.0.0:48081->8081/tcp   dockerbeamflink_jobmanager_1

henryp@corsair:~$ docker stop 87ec0891bc7b

87ec0891bc7b

henryp@corsair:~$ docker ps

CONTAINER ID        IMAGE                    COMMAND                  CREATED             STATUS              PORTS                                                                             NAMES

a057b8d8ff64        dataradiant/beam-flink   "/usr/local/flink/bi…"   20 hours ago        Up 20 hours         6123/tcp, 0.0.0.0:220->22/tcp, 0.0.0.0:48080->8080/tcp, 0.0.0.0:48081->8081/tcp   dockerbeamflink_jobmanager_1

whereupon the shell I had running in 87ec0891bc7b promptly terminates.

Hints on removing old images (they do take a lot of disk space) can be found at "Learn How To Stop, Kill And Clean Up Docker Containers"

Kubernetes

"Difference between Docker and Kubernetes: "Docker and rkt are two popular container technologies that allow you to easily run containerized applications. Kubernetes is a container orchestration platform that you can use to manage and scale your running containers across multiple instances or within a hybrid-cloud environment." (Google docs)