On May 21, Docker is discontinuing cluster management in Docker Cloud. This was disappointing at first, as I had an efficient deployment process using Stacks at Docker Cloud, but having gone through the process of migrating to deployments from my own machine I’m pretty happy with where I’ve ended up. The following notes might help others migrate.

The overall process is to use docker-machine to create a host for your containers, and docker stack deploy to deploy your stack. I’m running my docker host in Digital Ocean, which interfaces nicely with docker, though other cloud providers are available. I assume you have docker installed locally, as you’ll be running docker commands to manage the remote host.

We’re going to
– get an API key from Digital Ocean
– create a droplet running docker
– convert the stack configuration in Docker Cloud into docker compose format
– deploy the stack to our droplet

First step is to get an API key from Digital Ocean.

Log in to Digital Ocean, go to API, Tokens/Keys, Generate new token. Give your token a useful name, then copy it once it’s generated as there’s no way to get it once you leave this page.

With the API key, we can now create a droplet running docker and initialise a docker swarm (even if it’s a swarm of one). We’ll call the new host “docker-sandbox”

docker-machine create –driver digitalocean –digitalocean-access-token xxxxx –digitalocean-region sfo1 docker-sandbox

You’ll probably want to backup the folder at ~/.docker/machine/machines/docker-sandbox because it contains the keys you need to be able to ssh into your new droplet.

docker uses some environment variables to tell it to run commands on a particular host. You can see these using this command:

docker-machine env docker-sandbox

And eval that command in order to set the variables, eg if you’re working in a new bash session:

eval $(docker-machine env docker-sandbox)

Now we need to get the IP address of the newly created droplet:

docker-machine ls

With that, we can initialise a docker swarm, with the new droplet as the manager (xxxx is the IP address you just obtained)

docker-machine ssh docker-sandbox “docker swarm init –advertise-addr xxxx”

We have a swarm, so let’s get something running on it. We define our stack with a docker compose configuration file. Traditionally this is called docker-compose.yml. This is the file we’ll use:

version: "3.5"
services:

  mongo:
    image: 'mongo:latest'
    environment:
      - MONGO_INITDB_ROOT_USERNAME=mongo_username
      - MONGO_INITDB_ROOT_PASSWORD=mongo_password
    networks:
      private_network:
        aliases:
          - mongo
    ports:
      - '27017:27017'
    volumes:
      - '/data/db:/data/db'

  web:
    image: 'my_image_tag:latest'
    environment:
      - SSL_CRT=path_to_letsencrypt_fullchain.pem
      - SSL_KEY=path_to_letsencrypt_privkey.pem
    networks:
      - private_network
    ports:
      - '80:3000'
      - '443:3001'
    volumes:
      - '/var/webroot:/var/webroot'
      - 'path_to_letsencrypt:path_to_letsencrypt'
    depends_on:
      - mongo

networks:
  private_network:

Let’s go through it in detail.

At the high-level, we’re setting up two containers called mongo and web, and a network called private_network.

On lines 5 and 19 we specify the images for the two containers. Mongo uses the basic mongo image whilst web uses a bespoke image.

On lines 6-8 and 20-22 we define some environment variables for the root mongo user and the locations of certificates that the web container will need to support HTTPS.

On lines 9-12 and 23-24 we give each container access to a network called private_network that they’ll use to communicate with each other. The web container needs to be able to contact the mongo container, so lines 11-12 give the latter an alias “mongo” that can be used as a hostname in connection strings. The web container never needs to be contacted internally, so it doesn’t get an alias.

On lines 13-14 we expose port 27017 on the host and map it to the same port on the container. 27017 is the default mongo port, so this allows our mongo DB to be administered remotely. It’s a good idea to set up a separate firewall that limits the IP addresses that can connect to this port.

On lines 25-27 we expose ports 80 and 443 on the host, for HTTP and HTTPS respectively, and map them to ports 3000 and 3001 on the container. The web server within our container listens to web requests on those two ports.

On lines 15-16 and 28-30 we set up three paths within the containers that map to paths on the host. For the mongo container, this path stores the database, and it needs to store the database on the host so we don’t lose the database if the host restarts. For the web container, these paths are used by letsencrypt. During certificate generation, files are created in /var/webroot and accessed via the domain name over HTTP – proving to the letsencrypt server that we own the domain name. The freshly generated certificate is written to the host filesystem and the container must be able to read it from there when setting up the HTTPS server. It’s worth noting that all the host directories must exist before this stack can start up.

Finally, lines 31-32 specify that the web container depends upon the mongo container.

And that’s all there is to it!

Now all we need to do is to get the stack running on the host. Our web container is private so we need to log into docker so that future commands can access it:

docker login

And at last we are ready to deploy the stack:

docker stack deploy –with-registry-auth -c docker-compose.yml mystackname

We’ve used –with-registry-auth so that the host has the authentication details that we logged in with. docker-compose.yml is the configuration file for the stack. And mystackname is its name.

You should be able to check that it’s running

$ docker ps
CONTAINER ID        IMAGE                     COMMAND                  CREATED             STATUS              PORTS               NAMES
f222553ed53b        mongo:latest              "docker-entrypoint.s…"   4 minutes ago       Up 4 minutes        27017/tcp           mystackname_mongo.1.v5711gsrj8oidwikusynirwze
d2c8083be837        my_image_tag:latest       "/bin/sh ./server_ru…"   4 minutes ago       Up 4 minutes        3000-3001/tcp       mystackname_web.1.uj0m43k7mau16ukwcr4rap54g

And then visit your website in your browser.