Volumes and Persistent Data with Docker Compose

In this tutorial, you will learn the ins and outs of using volumes and persistent data with Docker Compose. This topic is crucial for anyone who wishes to maintain data across container lifecycle. You will learn about Docker volumes, how to use them in Docker Compose, and to solidify your understanding, I’ll walk you through an example of creating a persistent MySQL database. Let’s dive right in!

What are Docker volumes?

In the world of Docker, data is usually ephemeral, meaning it only exists for the lifespan of the container. However, what if we need our data to persist beyond the lifespan of a single container instance? That’s where Docker volumes come into play.

Docker volumes are essentially a mechanism that allows data to persist even after a container is stopped or deleted. They are like dedicated folders lying on the host filesystem that can be connected to one or more containers. This means any changes in this folder can be seen by all linked containers, providing a way to share data between containers. Furthermore, the data in the volume continues to exist even if no containers are currently using it.

How to use volumes in Docker Compose

Now that you understand what Docker volumes are, let’s see how to use them in Docker Compose.

Docker Compose is a tool that simplifies the process of managing multi-container Docker applications. It uses a YAML file (usually named docker-compose.yml) to define the multi-container setup.

To use a volume in Docker Compose, you need to define it in your docker-compose.yml file. Here is a basic example:

version: '3'
services:
  myservice:
    image: myimage
    volumes:
      - myvolume:/path/in/container
volumes:
  myvolume:

In this configuration, we’re creating a service named myservice using an image named myimage. The volumes field under the myservice specifies that a volume named myvolume should be mounted at /path/in/container within the running container. The volumes field at the root level of the configuration file declares the myvolume volume.

Example: Creating a Persistent MySQL Database

Now let’s put what you’ve learned into practice. I’ll walk you through an example of creating a persistent MySQL database using Docker Compose.

We’re going to create a docker-compose.yml file with the following content:

version: '3'
services:
  db:
    image: mysql:8
    volumes:
      - db_data:/var/lib/mysql
    environment:
      MYSQL_ROOT_PASSWORD: mysecretpassword

volumes:
  db_data:

Here we’re defining a single service named db using the mysql:8 image. The volumes field mounts a volume named db_data at /var/lib/mysql, which is where MySQL stores its data by default. We’re also setting an environment variable MYSQL_ROOT_PASSWORD to specify the root password for the MySQL server.

After you’ve created this file, you can start your MySQL container with Docker Compose by running the following command in the same directory as your docker-compose.yml file:

docker-compose up -d

The -d flag tells Docker Compose to run the containers in the background.

With this configuration, any data that your MySQL server writes to /var/lib/mysql will be stored in the db_data volume. If the container is stopped or deleted, the data will remain intact and can be used by new containers.

To verify that the data is persisting, you could create a table in the MySQL database, then stop and remove the containers with docker-compose down, and then bring them back up again with docker-compose up -d. After doing this, you should see that the table you created still exists, which means your data has persisted.

How can I view the data inside a Docker volume?

Now that you’re familiar with Docker volumes and how to use them with Docker Compose, you might be wondering, “How can I actually view the data inside a Docker volume?” In this section, I will guide you through the steps to do exactly that.

Firstly, it’s important to understand where Docker volumes are stored. On a typical Linux installation, Docker stores the data in volumes in the following directory on the host machine: /var/lib/docker/volumes/. Each volume is represented by a directory within this folder. However, the data is not easily accessible and it’s not recommended to directly interact with these directories, as Docker manages them.

So, how can we view the data? The answer is by creating a new Docker container that has the volume attached, and then exploring the data from within this new container.

Let’s say we have a volume named myvolume and we want to view its contents. We can create and run a temporary container which mounts myvolume. For this purpose, we can use a lightweight image like busybox. Here’s the command:

docker run -it --rm -v myvolume:/vol busybox ls /vol

Here’s what’s happening in this command:

docker run is the command to create and start a new container.
-it tells Docker to run the container in interactive mode so you can interact with the BusyBox shell.
--rm tells Docker to remove the container after it exits. This is useful for cleanup since we’re only creating this container to peek into the volume.
-v myvolume:/vol mounts the myvolume volume to the path /vol inside the container.
busybox is the name of the image we’re using to create the container. BusyBox is a lightweight Linux distribution that’s useful for debugging and exploring.
ls /vol is the command we’re running inside the new container. This will list the contents of the /vol directory, which is where we’ve mounted myvolume.

After you run this command, you should see a list of files and directories. These are the contents of your myvolume volume.

Remember, although it’s technically possible to view the data in a Docker volume directly from the host filesystem, it’s generally not recommended because Docker volumes are intended to be managed by Docker. Interacting with them directly could lead to unexpected results. Instead, using a Docker container to explore the volume’s contents, as described above, is the preferred method.

I hope this helps you understand how to view the data inside a Docker volume. With this knowledge, you can now confidently manage your data in Docker!

How can I backup Docker volumes?

As you work with Docker volumes, one question that might come up is, “How can I backup Docker volumes?” Given the importance of data persistence and protection, this is a crucial aspect to understand. In this section, I’ll guide you through the process of backing up Docker volumes.

While Docker does not provide an out-of-the-box solution for backing up volumes, there are several strategies you can employ to protect your data. We’re going to focus on one of the most common methods: using a temporary Docker container to create a backup of the volume’s data.

Let’s assume we have a Docker volume named myvolume that we want to backup. The first step is to create a temporary Docker container that mounts the volume. We can then use this container to create a tarball (a type of compressed file) of the volume’s data. Here’s the command to do that:

docker run --rm -v myvolume:/volume -v $(pwd):/backup alpine tar -czf /backup/myvolume-backup.tar.gz -C /volume ./

This command might seem complex, but let’s break it down:

docker run is the command to create and start a new container.
--rm tells Docker to remove the container after it exits. This is useful because we’re only creating this container to make the backup.
-v myvolume:/volume mounts the myvolume volume to the path /volume inside the container.
-v $(pwd):/backup mounts the current directory ($(pwd)) on the host to the path /backup inside the container. This is where the backup file will be created.
alpine is the name of the image we’re using to create the container. Alpine Linux is a lightweight Linux distribution that’s useful for tasks like this.
tar -czf /backup/myvolume-backup.tar.gz -C /volume ./ is the command we’re running inside the new container. This command creates a gzipped tarball of the contents of the /volume directory and writes it to /backup/myvolume-backup.tar.gz.

After you run this command, you’ll find a file named myvolume-backup.tar.gz in the current directory of your host machine. This file is a compressed archive of your volume’s data.

To restore the volume from the backup, you can use a similar method. Here’s the command to restore the volume from the backup:

docker run --rm -v myvolume:/volume -v $(pwd):/backup alpine tar -xzf /backup/myvolume-backup.tar.gz -C /volume

This command is almost identical to the previous one. The key difference is that we’re using tar -xzf to extract the tarball instead of tar -czf to create it.

I hope this section has provided you with a clear understanding of how to backup and restore Docker volumes. Remember, regular backups are an essential part of maintaining data integrity and preventing data loss.

Conclusion

In this tutorial, you’ve learned about Docker volumes, how to use them with Docker Compose, and you’ve seen an example of how to create a persistent MySQL database. I hope you found this information useful and feel more comfortable with these concepts now. Remember, Docker volumes are a powerful tool for managing and preserving data in your Docker applications. Happy Dockering!