Programster's Blog

Tutorials focusing on Linux, programming, and open-source

Docker Compose Cheatsheet

cheatsheet Docker docker-compose

Docker Compose File
Commands
Build Arguments
Environment Variables
Health Check
References

Docker Compose File

Compose is a tool for defining and running multi-container Docker applications. It utilizes a YAML configuration file (usually called docker-compose.yml) to specify the services and how to launch them.

One can find the official documentation for Docker Compose online.

Example Configuration File

This is a sample docker-compose.yml configuration file that should hopefully help get you started. It is easier to remove things than to find what to add.

services:
  app:
    build:
      context: .
      dockerfile: Dockerfile
    container_name: app
    image: ${REGISTRY}/my-project-name
    restart: always
    ports:
      - "80:80"
      - "443:443"
    depends_on:
        - db
    env_file:
      - ./.env

  db:
    image: mariadb:10.5
    container_name: db
    restart: always
    volumes:
      - mysql-data:/var/lib/mysql
    environment:
      - FOO=bar
      - SOME_ENV_VAR=${SUBSTITUTED_VARIABLE}
    env_file:
      - ./.env

volumes:
  mysql-data:
    driver: local

I used to provide the following older configuration file which is below for reference, but have since changed it because a few things have changed since older versions, such as:

One no longer needs to specify a network backend for services to communicate if you only the most simple setup where all containers can talk to all other containers in the docker compose file. This is because docker compose will set this up by default now, with all containers using a single backend. However, if you wish to prevent containers specified in a single docker compose file from being able to talk to each other, then you would need to set up the relevant backend networks.
One no longer needs to specify the version in docker compose files. Only in older versions is this required.

version: "3"

networks:
  backend:
    driver: bridge

services:
  app:
    build:
      context: .
      dockerfile: Dockerfile
    container_name: app
    image: ${REGISTRY}/my-project-name
    restart: always
    ports:
      - "80:80"
      - "443:443"
    depends_on:
        - db
    networks:
      - backend
    env_file:
      - ./.env

  db:
    image: mariadb:10.5
    container_name: db
    restart: always
    networks:
        - backend
    volumes:
      - mysql-data:/var/lib/mysql
    environment:
      - FOO=bar
      - SOME_ENV_VAR=${SUBSTITUTED_VARIABLE}
    env_file:
      - ./.env

volumes:
  mysql-data:
    driver: local

Volumes

Bind Mount Volumes

A bind-mount volume is one where you specify a folder locally that should be mounted within the container at a certain path. Thus one needs to specify the local directory path, and the path it will be within the container like so:

version: "3"

services:
  db:
    image: mariadb:10.5
    container_name: db
    volumes:
      - /some/local/dir:/var/lib/mysql

Read Only Volume

If you don't want the container to be able to change the data within the volume, such as if it is a config file or set of TLS certificates that should not change, then you can specify that it is a "readonly" by simply adding :ro to the end like so:

version: "3"

services:
  db:
    image: mariadb:10.5
    container_name: db
    volumes:
      - /some/local/dir:/var/lib/mysql:ro

Long Form

Having the volume specified in one line is the "short form". If you want a more verbose declaration, then you can break it out over multiple lines like so:

version: "3"

services:
  db:
    image: myAppImage:latest
    container_name: app
    volumes:
      - type: bind
        source: /some/local/dir/ssl
        target: /etc/nginx/ssl
        read_only: true

Named Volumes

If your container just needs to persistently store state that it generates, then named volumes are great. You can use named volumes like so:

version: "3"

services:
  db:
    image: mariadb:10.5
    container_name: db
    volumes:
      - my-volume-name:/var/lib/mysql

volumes:
  my-volume-name:
    driver: local

The long-form declaration for a named volume is like so:

version: "3"

services:
  db:
    image: mariadb:10.5
    container_name: db
    volumes:
      - type: volume
        source: my-volume-name
        target: /var/lib/mysql

volumes:
  my-volume-name:
    driver: local

Specifying Hosts

If you need to manually pass DNS records to your containers (becuase updating your local server's /etc/hosts file won't work), then you can do that like so:

    extra_hosts:
      - "subdomain.mydomain.org:192.168.1.123"

e.g.

services:
  kibana:
    image: docker.elastic.co/kibana/kibana:7.9.0
    extra_hosts:
      - "elastic-search.programster.org:192.168.1.123"

Restart Choices

If you specify no for your restart (which you don't actually need to do because it's the default), then be aware that it is the only value you need to wrap in quotation marks. The choices are:

"no"
always
on-failure
unless-stopped

Relevant docs area.

Logging

Disable Logging

If you want to disable output from one of the services, just set the logging driver to none, like so:

  mysql:
    image: mysql:5.6
    restart: "no"
    logging:
      driver: none

Logging Drivers

In the previous example, to disable logging, you saw that we set the driver to none. Below are the logging drivers that I find most commonly useful:

none - don't log
local - logs to a local file, in a manner optimized for logging performance. Defaults to a total storage of 100MB spread over 5 x 20MB files. More info.
json-file - This is the default logging driver that Docker will use. This will capture the stdout and stderr output of your containers and writes them to a file using JSON format. This format annotates each line with its origin (stdout and stderr) and its timestamp. More info.
journald - sends container logs to the systemd journal. Log entries can be retrieved using the journalctl command, through use of the journal API, or using the docker logs command. More info.

View the full list of available logging drivers in the official documentation.

Logging Options

Each of the drivers can have options specified. One would expect to only need these options if using the local or json-file drivers, which each come with the following options:

max-size - the maximum size of the logging file, such as "50m".
max-file - the maximum number of files to keep (rotating out the oldest ones when the limit is reached)
compress - whether to compress these files or not. Defaults to enabled for the local driver, and disabled for the json-file driver.

There are additional options for the json-file driver, with different defaults to the local driver. Be sure to read about the options here:

JSON File Log Driver Example

Below is an example configuration where we tell Docker Compose to perform log rotation across 10 files that are each up to 50MB in size, and to compress the non-active log files.

version: "3"

services:
  db:
    image: mariadb:10.5
    logging:
      driver: "json-file"
      options:
        max-size: "50m"
        max-file: "10"
        compress: "true"

Networking

Use Host Networking

Below is a minimal example where we specify that the container should use the host network.

services:
  app:
    image: 'tutum/hello-world'
    network_mode: host

Using the host network will mean that:

Whatever ports the container wants to use, will be bound directly on the host,
There is no point using the ports declaration for the container in the docker compose file when doing this.
You have the risk that you may be exposing ports that you are not aware of (why not check with the ss command?).
One cannot remap the ports, such as mapping port 8080 or 3000 to the host's port of 80.

However, using host networking may be required if one needs to make use of custom iptables/nftables firewall rules and not have docker manipulate the iptables rules.

Create Backend Networks

If no backend networks are specified, then docker compose will create a single backend network that all of the containers can communicate with each other on, but containers from other docker compose files will not be able to communicate with, unless you specify that the network is external.

Below is a simple example that allows webapp1 to communicate with webapp1db and webapp2, but webapp2 cannot communicate with webapp2 db (but can with webapp1).

networks:
  webApp1Backend:
    driver: bridge
  crossAppCommsBackend:
    driver: bridge

services:
  webapp1:
    build:
      context: .
      dockerfile: Dockerfile
    container_name: app
    image: ${REGISTRY}/my-project-name
    restart: always
    ports:
      - "80:80"
      - "443:443"
    depends_on:
        - db
    networks:
      - webApp1Backend
      - crossAppCommsBackend
    env_file:
      - ./.env

  webapp1db:
    image: mariadb:10.5
    container_name: db
    restart: always
    networks:
        - webApp1Backend
    volumes:
      - mysql-data:/var/lib/mysql
    environment:
      - FOO=bar
      - SOME_ENV_VAR=${SUBSTITUTED_VARIABLE}
    env_file:
      - ./.env

  webapp2:
    build:
      context: .
      dockerfile: Dockerfile
    container_name: app
    image: ${REGISTRY}/my-project-name
    restart: always
    ports:
      - "800:80"
      - "4430:443"
    networks:
      - crossAppCommsBackend
    env_file:
      - ./.env

volumes:
  mysql-data:
    driver: local

External Backend Network

If you need to have your containers connect to a backend network that already exists and was probably manually set up, then you just need to specify that it is an "external" backend. This is commonly used for things like Traefik, where one sets up a traefik network, and then has other services connect onto that network. You can see this in action at the point of creating the network in my Getting started with Traefik tutorial.

networks:
  traefik-net:
    external: true

services:
  reverse-proxy:
    container_name: traefik
    image: traefik:v2.8
    networks:
      - traefik-net
    ports:
      - "80:80"
      - "8080:8080"
    volumes:
      - ./static-config.yml:/etc/traefik/traefik.yml:ro # Provide the config file.
      - /var/run/docker.sock:/var/run/docker.sock # So that Traefik can listen to the Docker events

Commands

Launch Everything

Start all the containers/services using docker-compose

docker-compose up

Start A Specific Service

Sometimes one needs to only start a specific service, or manually start the services in order to start them in the correct order. E.g. it's nice to start your database service first so it's ready when your application starts.

docker-compose up $SERVICE_NAME

Specify Docker Compose File

docker-compose will use a file called docker-compose.yml by default, but if you want to use a different file, you can specify its path.

docker-compose -f /path/to/file.yml up

I use this when I have a separate docker-compose file for dev, which will use a volume that production doesn't use.

Execute Command In A Running Service

The following can be used to run a command against a service (not the name of the container).

docker-compose exec $SERVICE_NAME /command/to/run param1

e.g.

docker-compose exec web php artisan october:up

Specify Compose File

Specify a custom filepath for your docker-compose file (it assumes docker-compose.yml in your current directory by default)

docker-compose -f custom-docker-compose.yml up

Apply multiple compose files (changes in latter)

docker-compose -f docker-compose.yml docker-compose-production.yml

Re-deploy just one service. Particularly useful after a rebuild.

docker-compose up $SERVICE_NAME

Environment Variables

There are many ways to pass environment variables to your container through the use of docker-compose.

Individually Inside The Docker Compose File

Firstly, you can manually specify them individually like so:

version: "3"

services:
  app:
    image: my-docker-image
    environment:
      - MY_VARIABLE_NAME="bob"
    ports:
      - "80:80"

This is a pretty safe route, but means that everyone will be using the same environment variables, which is probably not what you want if you are tracking the docker-compose.yml file as part of your source control.

Implicit `.env` File

You can have a .env file at the same location as your docker-compose file. All variables within this .env file will automatically be used by docker-compose.

I need to check/confirm if it is same location as docker-compose.yml file, or path of where we are calling docker-compose from which may be different). E.g If one was to run docker-compose -f /path/to/docker-compose.yml up.

Values in the shell take precedence over those specified in the .env file.

Specify Env File For Docker Compose

docker-compose --env-file ./config/.env up

Values in the shell take precedence over those specified in the .env file.

Specify .env File In Compose File

Use the .env_file declaration as part of your configuration to specify where the .env file is that you wish to use.

version: "3"

services:
  app:
    container_name: app
    image: my-image-name
    env_file:
      - ./path/to/.env
    ports:
      - "80:80"

This is the route I most commonly use:

Values in the shell take precedence over those specified in the .env file.

Variables In Compose File

If you need to use a variable in the docker-compose file, use the ${VARIABLE_NAME} syntax and provide the VARIABLE_NAME through either an environment variable, or an ARG (details of the many various ways of providing environment variables and arguments detailed above).

For example, the configuration below will use environment or ARG variables to build the image declaration using the DOCKER_REGISTRY, PROJECT_NAME, AND TAG_VERSION variables.

version: "3"

services:
  app:
    build:
      context: .
      dockerfile: ./Dockerfile
    container_name: app
    image: ${DOCKER_REGISTRY}/${PROJECT_NAME}:${TAG_VERSION}
    restart: always
    ports:
      - "80:80"

Default Value For Variable

One can take this a step further, and specify a default value if the environment variable does not exist.
For example, the following docker-compose file sets the env_file to .env if there is no $ENVFILE set.

version: "3.7"
services:
  app:
    image: ubuntu:22.04
    env_file: ${ENVFILE:-.env}

Docker Compose Variables

Compose Project Name

It's worth setting the COMPOSE_PROJECT_NAME environment variable for specifying what I call the "namespace" of your project. This will add prefixes, to try and prevent generic names for things like your volumes, clashing with other projects. E.g. if you have two projects that use a MySQL volume called "mysql-data", you wouldn't want them to both reference the same volume.

If you don't set this variable, prefixes will be based on the folder name of where your docker-compose.yml file is.

You can also manually specify the project name as a parameter when calling docker-compose like so:

docker-compose --project-name "my-project-name` up

...or:

docker-compose -p "my-project-name` up

Official documentation.

Build Arguments

Please refer here.

Health Check

Docker Compose has supported health check configuration since the 3.4 file format. You can read about how docker uses the health checks here.

Below is an example where our web app is dependent on a PostgreSQL database and a redis server, which both have health checks configured, so the web app won't spin up until both of those services are ready.

version: '3.4'
services:
  db:
    restart: always
    image: postgres:14-alpine
    healthcheck:
      test: ['CMD', 'pg_isready', '-U', 'postgres']
    volumes:
      - ./postgres14:/var/lib/postgresql/data
    environment:
      - 'POSTGRES_HOST_AUTH_METHOD=trust'

  redis:
    restart: always
    image: redis:7-alpine
    healthcheck:
      test: ['CMD', 'redis-cli', 'ping']
    volumes:
      - ./redis:/data

  web:
    image: programster/example-app
    restart: always
    ports:
      - "80:80"
    depends_on:
      - db
      - redis
    environment:
      - FOO=bar

References

Last updated: 6th August 2024
First published: 23rd August 2020

Programster's Blog

Tutorials focusing on Linux, programming, and open-source

Docker Compose Cheatsheet

Related Posts

Table Of Contents

Docker Compose File

Example Configuration File

Volumes

Bind Mount Volumes

Read Only Volume

Long Form

Named Volumes

Specifying Hosts

Restart Choices

Logging

Disable Logging

Logging Drivers

Logging Options

JSON File Log Driver Example

Networking

Use Host Networking

Create Backend Networks

External Backend Network

Commands

Launch Everything

Start A Specific Service

Specify Docker Compose File

Execute Command In A Running Service

Specify Compose File

Environment Variables

Individually Inside The Docker Compose File

Implicit .env File

Specify Env File For Docker Compose

Specify .env File In Compose File

Variables In Compose File

Default Value For Variable

Docker Compose Variables

Compose Project Name

Build Arguments

Health Check

References

Implicit `.env` File