CSI in Docker Swarm!

⚠️ This article is an automated translation. While I personally reviewed the content before publication, some inaccuracies may remain. Read the original French version.

With version 23, Docker (actually moby/swarmkit) added CSI support in Swarm mode. This feature was long-awaited within the (small) Swarm community, as it had been its biggest drawback compared to Kubernetes until then (if we only talk about Swarm’s native features, not the surrounding ecosystem).

What is CSI?

CSI stands for Container Storage Interface. The concept is to standardize how container orchestrators (Docker Swarm, Kubernetes, Cloud Foundry, etc.) handle access to persistent storage (on disk, for example). Indeed, when you use an orchestrator, it means your containers can start on several different machines. However, a major problem arises: how to handle containers that need to store persistent data, since these containers must be able to start on any of the machines in your cluster?

CSI harmonizes how orchestrators do this (much like CNI with the network), and allows you to directly declare how the instantiated containers should access the data: locally, on an NFS mount, Samba, Seaweed, Gluster, S3, etc. The orchestrator then takes care of making the mount point directly in the container, which means the host system does not have to access the mount itself.

Apart from practicality (data is mounted the same way as a virtual network or a configuration), it is also a gain in reliability (no risk of forgetting a mount point on the host that propagates poorly into a container, or a machine losing the mount without realizing it), and security (system hosts do not have direct access to the data, which limits file permission issues and prevents anyone from reading the data).

In short, the Container Storage Interface is a standard that brings real comfort of use daily, significantly improving the quality of our infrastructure. While Kubernetes users have been benefiting from it since 2019, Docker Swarm users had to wait until February 2023 to see this feature arrive in experimental status (in other words, it might all go black, so take it easy before using it in production!).

Swarm CSI - How Does It Work?

Important Note: Due to a bug on Docker Swarm, this tutorial does not work on Docker versions 26.0.2 to 28.2.0.

Docker Swarm comes with CSI, but without an integrated provider: users must install plugins themselves on each of the nodes in the Swarm cluster for their use. And where it hurts a bit is that Mirantis (the main contributor to Swarmkit and seller of Swarm Enterprise solutions) has carefully avoided providing ready-to-use plugins. The community has therefore started to take up the subject. Some repositories like this one bring together documentation and some drivers. Among those supported, we can mention NFS, Samba, SeaweedFS, and some Cloud Providers that have ported their Kube plugin to Swarm (Hetzner and DigitalOcean are among the good students).

Once the plugin is installed on each of your Docker hosts, you will be able to create Swarm services using mount points through these plugins.

Putting It into Practice

In this example, we will use four virtual machines located on the same virtual network (10.0.0.0/8): three machines form a Docker Swarm cluster (10.0.1.1, 10.0.1.2, 10.0.1.3), and one machine acts as an NFS server (10.0.100.1). These four machines are installed under Debian 12.

Installing the NFS Server

First, let’s connect to our NFS machine and install the nfs-kernel-server package; and create the folder /mnt/myshareddir.

Then, edit the /etc/exports file and put this content:

/mnt/myshareddir 10.0.1.0/24(rw,sync,no_subtree_check)

Finally, export the new configuration and restart the NFS server: exportfs -a ; systemctl restart nfs-kernel-server.

Your NFS server is created; we can move on to the cluster. Of course, this configuration is very basic (no authentication, basic control via IP…). It goes without saying that this configuration is not suitable for production.

Installing the Swarm Cluster

Install Docker following the official documentation on each machine. Due to an unresolved bug in version 27 (see the top of the article), you will need to go back to version 26.0.2 (on each machine as well):

apt install -y docker-ce=5:26.0.2-1~debian.12~bookworm --allow-downgrades

Then create a Swarm cluster. The detailed version is available here, but for those in a hurry, here is a small tl;dr:

  1. On the first machine, type docker swarm init ; docker swarm join-token manager
  2. On the other two machines, type the command obtained in the previous step.

Our cluster is operational! A quick docker node ls will allow you to make sure.

Now, we need to install the CSI-NFS plugin that will allow us to use our NFS server directly in Swarm. Since the latest version of the plugin was not available, I compiled and pushed it to DockerHub myself, based on the sources present here.

On each machine, type:

docker plugin install \
  --alias csi-nfs \
  --grant-all-permissions \
  artheriom/swarm-csi-nfs:v4.9.0

If everything went well, you should have a result similar to this on each of the machines by typing docker plugin ls:

root@test-1:/home/debian# docker plugin ls
ID             NAME             DESCRIPTION                       ENABLED
4af20db19305   csi-nfs:latest   NFS CSI driver for Docker Swarm   true

Creating Our Volume

Now that everything is ready, we can create a new volume on our cluster, which will use our NFS server! This command should be run on a single node (the volume will propagate throughout the cluster):

docker volume create \
  --driver csi-nfs \
  --type mount \
  --opt server="10.0.100.1" \
  --opt share="/mnt/myshareddir" \
  my-nfs-volume

Here, we are creating a volume named my-nfs-volume of type mount (which makes it explicit to Swarm that in reality, we are using a mount point), using our driver installed via the plugin; and which will be created on our NFS server. There are several other configuration options not covered here, feel free to consult the documentation if needed.

If everything worked well, your volume should have the status Created:

root@test-1:/home/debian# docker volume ls --cluster
VOLUME NAME               GROUP     DRIVER         AVAILABILITY   STATUS
my-nfs-volume                       csi-nfs        active         created

If your volume remains in pending creation, check the configuration of your NFS server.

Using Our Mount!

Now that your volume is created, you can use it in your service!

docker service create \
  --name my-service \
  --mount type=cluster,src=my-nfs-volume,dst=/usr/share/nginx/html \
  --mode global \
  nginx

And there you go, you are using your first mount via CSI for your data! 🎉 You can check that everything is working well by adding/deleting content in the containers and checking that the changes propagate correctly.

Docker Swarm CSI - In the End?

Swarm’s CSI is like the rest of Swarm: it’s simple, elegant, and functional. The modular side by plugin allows almost infinite extensibility and is totally adjustable according to needs, which is another excellent news.

However, this technology is still experimental, and it shows:

In short, and to conclude, CSI in Swarm is excellent news for the life of the product; but a long way remains before having this kind of feature usable in real conditions. And that’s a shame, for a feature that should have been there… In 2020.