Scale Docker Compose

Purpose

Level: easy.

The purpose of this post is to review how we can scale Docker services using Docker Compose version 3 without the need of using Docker swarm. These are some possible scenarios:

  • Load testing launching more than one test at a time.
  • Network testing using multiple replicas and Docker networks.
  • Deploy multiple services behind a load balancer.

Before you begin

In this tutorial, we’ll learn how to increase the number of Docker replicas using Docker Compose. A Linux/Mac OS/Windows machine with Docker and Docker Compose (compatible with version 2: Compatibility-matrix ) installed is required to follow this tutorial.

Deployment without Docker replication

Docker-compose example without scaling any Docker containers:

version: '3'

networks:
    my_net:

services:

    star:
        image: python:alpine
        command: "sh -c \"echo This is a star with hostname: $$HOSTNAME && sleep 365d\""
        networks:
          - my_net

    planet:
        image: python:alpine
        command: "sh -c \"echo This is a planet with hostname: $$HOSTNAME && sleep 365d\""
        networks:
          - my_net

    satellite:
        image: python:alpine
        command: "sh -c \"echo This is a satellite with hostname: $$HOSTNAME && sleep 365d\""
        networks:
          - my_net

Command to start previous containers:

docker-compose up

Result:

$ docker-compose up
Stopping and removing dev_satellite_2 ... done
Stopping and removing dev_satellite_3 ... done
Stopping and removing dev_planet_2    ... done
Starting dev_star_1                   ... done
Starting dev_satellite_1              ... done
Starting dev_planet_1                 ... done
Attaching to dev_star_1, dev_planet_1, dev_satellite_1
star_1       | This is a star with hostname: 5cf5989c64b5
planet_1     | This is a planet with hostname: 8b4ec23e7194
satellite_1  | This is a satellite with hostname: 2b02218451a3

As you can see in the previous example we’ve deployed one container for each service.

Scaling Docker containers

With the following command we are scaling the satellite service to three replicas and the planet to two replicas.

docker-compose up --scale satellite=3 --scale planet=2

Result:

$ docker-compose up --scale satellite=3 --scale planet=2
Starting dev_star_1      ... done
Starting dev_satellite_1 ... done
Starting dev_satellite_2 ... done
Starting dev_satellite_3 ... done
Starting dev_planet_1    ... done
Starting dev_planet_2    ... done
Attaching to dev_star_1, dev_satellite_1, dev_satellite_2, dev_satellite_3, dev_planet_1, dev_planet_2
satellite_1  | This is a satellite with hostname: 2b02218451a3
star_1       | This is a star with hostname: 5cf5989c64b5
satellite_2  | This is a satellite with hostname: f3bde6890623
satellite_3  | This is a satellite with hostname: 29c1b30f190f
planet_1     | This is a planet with hostname: 8b4ec23e7194
planet_2     | This is a planet with hostname: 2f13ce4d3c13

As you can see we can easily scale the number of replicas of a service using --scale with docker-compose command. Finally, we can ping one container from another using the hostname (same as container id) or by docker name like dev_satellite_N for example.

If we want to modify the number of replicas once the containers are running we can use docker-compose scale command. In this example we’ll reduce to the initial number of containers that were running at the beginning of the post, to one each.

docker-compose scale star=1 planet=1 satellite=1

See Docker Compose scale reference for more info.

Finally, you should definitely take a look at these books to fuel your Docker knowledge: