A quick and painless introduction to Docker and the features it provides and why you should probably use it, with some examples. In other words, why you should use it.
Why do we need virtualization?
Applications in general need to interact with the operating system, the hardware, and other applications.
If even an element of the system changes, the application may not work as expected.
Virtual machines
Virtual Machines (VMs) are a great way to improve reproducibility.
They use a very complex software, the hypervisor, to emulate the hardware and the operating system. They are (mostly) completely independent from the host system they are running on.
Containers
Containers are a lightweight alternative to VMs.
They use some features of the Linux kernel, namely namespaces and cgroups, to completely isolate a process from the rest of the system. They can’t emulate different kernel or hardware.
Docker
It also provides a registry where you can find pre-built images.
Installation
Since Docker is specifically built with the Linux kernel in mind, it is not natively supported on Windows and macOS. It needs to spin up a virtual machine to run Linux.
Docker Desktop handles everything for you.
- Windows
- Mac
- Linux
- Docker Desktop (will use a VM)
- Docker Engine
Images
An image is a read-only template with instructions for creating a Docker container.
They are made up of layers that are stacked on top of each other, like git commits.
Only the top layer is writable.
Dockerfile
Dockerfiles are the instructions to build an image.
FROM python:3.9.7-slim-buster # Base image
WORKDIR /app # Working directory (creates it if it does not exist)
COPY requirements.txt requirements.txt # Copy files from host to container
RUN pip install -r requirements.txt # Run a command in the container
COPY . . # Copy files from host to container
ENTRYPOINT ["python", "main.py"] # Run a command as soon as the container starts
Build an image
# Build an image from the Dockerfile in the current directory
# docker build -t <image-name>:<tag> <context>
# -t: tag the image with a name
# .: use the current directory as context
docker build -t my-image:latest .
Run a container
# Run a container from an image
# docker run <image-name>:<tag>
# -d: run the container in detached mode
# -p: publish a container's port(s) to the host
# --name: name the container
# --rm: remove the container when it exits
docker run -d -p 5000:5000 --name my-container --rm my-image:latest
Storage
Storage inside a container is ephemeral and is destroyed when the container is removed.
Volumes and bind mounts are used to persist data.
Transferring files into containers
- Copy them directly into the image. The files become part of the image and are independent from the host.
# COPY <src> <dest>
# ADD has the same syntax, src can be a URL or a tar file
COPY requirements.txt requirements.txt
COPY . .
- Use a bind mount. In this case, the connection is bidirectional and continuous.
# docker run -v <host-path>:<container-path>
docker run -v $(pwd):/app
Ports
Ports are used to expose a container’s network socket to the host.
This must be done explicitly when running the container with the -p flag.
Port visualization
Loading diagram...
# docker run -p <host-port>:<container-port> <image-name>
docker run -p 8000:80 my-web
docker run -p 3306:3306 my-database
Docker compose
Docker compose is a tool for defining and running multi-container Docker applications.
It uses a declarative YAML file to define and configure all the different services that make up the application. It can also specify volumes and networks to be created automatically as needed.
Even when dealing with a single container, having a docker-compose.yml file allows to easily visualize and change its configuration without having to use all the command line frags.
docker-compose.yml
version: "3.9" # Version of the docker-compose file
services: # List of services
db:
image: mysql # Pull the image from Docker Hub
volumes:
- db-data:/var/lib/mysql # Create a volume to persist data
environment: # Set environment variables
MYSQL_ROOT_PASSWORD: example
MYSQL_DATABASE: example
MYSQL_USER: example
MYSQL_PASSWORD: example
web: # Name of the service
build: . # Build the image from the Dockerfile in the current directory
ports: # List of ports to forward
- "5000:5000"
volumes: # List of volumes to mount
- .:/app
depends_on: # List of services to start before this one
- db
volumes: # List of volumes
db-data: # Name of the volume
Examples
Three examples of how to use Docker in your workflow.
- Distributing a Haskell script
- Running Qiskit in a Jupyter Notebook
- Web architecture with php and mysql
Distributing a Haskell script
You have a Haskell script that you want to distribute to other people.
They may not have the whole Haskell toolchain installed or they may be unfamiliar with it.
Docker provides easy portability.
# https://hub.docker.com/_/haskell
FROM haskell:slim # Official base image
COPY main.hs . # Copy the script into the container
RUN ghc -o main main.hs # Compile the script
ENTRYPOINT ["./main"] # Run the script as soon as the container starts
docker build -t my-haskell-script . # Build the image
docker run -it --rm my-haskell-script # Create and run the container
Running Qiskit in a Jupyter notebook
You want to run a Jupyter Notebook with Qiskit on a machine that does not have nor want Python and the right packages installed. Docker provides immediate access to the right environment without polluting the host system.
# https://quay.io/repository/jupyter/datascience-notebook
FROM quay.io/jupyter/datascience-notebook # Official base image
USER root # Switch to root user
COPY requirements.txt . # Copy the requirements file
RUN pip install --no-cache-dir -r requirements.txt # Install the requirements
docker build -t jupyter-qiskit-example . # Build the image
docker run -it --rm \ # Create and run the container.
-p 8888:8888 \ # Forward port 8888 to the host
-v $(pwd)/notebooks:/home/jovyan/work \ # Bind mount the notebooks directory
-e NB_UID=$(id -u) -e NB_GID=$(id -g) -e GRANT_SUDO=yes \ # Env vars
jupyter-qiskit-example
Web architecture with php and mysql
You want to run a php application with a mysql database.
Loading diagram...
docker compose up -d # Run the application in detached mode
docker compose down # Stop the application
docker volume prune # Remove all unused volumes
Commands
A collection of the most common commands you may need
Images commands
# Build an image
docker build -t <image-name>:<tag> <context>
# List all images
docker images
# Remove an image
docker rmi <image-name>:<tag>
# Remove all images
docker rmi $(docker images -q)
# Remove all dangling images
docker image prune
Container creation commands
# Create and run a container from an image
# docker [OPTIONS] run <image-name>:<tag>
# (some) OPTIONS:
# -it: run the container in interactive mode with terminal attached
# -d: run the container in detached mode
# -p: publish a container's port(s) to the host
# --name: name the container
# --rm: remove the container when it exits
# -v <host-path>:<container-path>: mount a host directory to a container directory
# -v <volume-name>:<container-path>: mount a volume to a container directory
# --env-file <file>: load environment variables from a file
# -e <key>=<value>: set an environment variable
# --entrypoint <command>: run a command as soon as the container starts
docker run -d -p 5000:5000 -v my-volume:/app --name my-container my-image:latest
docker run -it -v ./app:/app my-image:latest
docker run -e MYSQL_PASSWORD=pass --rm my-image:latest
docker run -it --entrypoint "/bin/bash" my-image:latest
Container management commands
# List all running containers
docker ps
# List all containers
docker ps -a
# Stop a container
docker stop <container-name>
# Stop all running containers
docker stop $(docker ps -q)
# Remove a container
docker rm <container-name>
Container interaction commands
# Run a command in a running container
docker exec <container-name> <command>
# Attach a terminal to a running container
docker exec -it <container-name> /bin/bash
# Show logs of a container
docker logs <container-name>
# Show logs of a container in real time
docker logs -f <container-name>
Volume commands
# Create a named volume
docker volume create <volume-name>
# List all volumes
docker volume ls
# Remove a volume
docker volume rm <volume-name>
# Remove all unused volumes
docker volume prune
Docker compose commands
# Run the application
docker compose up -d
# Force rebuild of images
docker compose up --build
# Run a single container
docker compose up <service-name>
# Stop the application
docker compose down
# List all running containers
docker compose ps