rancherOS/docs/os/storage/using-zfs/index.md

---

title: Using ZFS in RancherOS layout: os-default redirect_from:

• os/configuration/storage/ ---

Using ZFS

Installing the ZFS service

The zfs service will install the kernel-headers for your kernel (if you build your own kernel, you'll need to replicate this service), and then download the [ZFS on Linux]() source, and build and install it. Then it will build a zfs-tools image that will be used to give you access to the zfs tools.

The only restriction is that you must mount your zpool into /mnt, as this is the only shared mount directory that will be accessible throughout the system-docker managed containers (including the console).

$ sudo ros service enable zfs
$ sudo ros service up zfs
# you can follow the progress of the build by running the following command in another ssh session:
$ sudo ros service logs --follow zfs
# wait until the build is finished.
$ lsmod | grep zfs

Note: if you switch consoles, you may need to re-run ros up zfs.

Using ZFS

After it's installed, it should be ready to use. Make a zpool named zpool1 using a device that you haven't yet partitioned (you can use sudo fdisk -l to list all the disks and their partitions).

Note: You need to mount the zpool in /mnt to make it available to your host and in containers.

$ sudo zpool list
$ sudo zpool create zpool1 -m /mnt/zpool1 /dev/<some-disk-dev>
$ sudo zpool list
$ sudo zfs list
$ sudo cp /etc/* /mnt/zpool1
$ docker run --rm -it -v /mnt/zpool1/:/data alpine ls -la /data

To experiment with ZFS, you can create zpool backed by just ordinary files, not necessarily real block devices. In fact, you can mix storage devices in your ZFS pools; it's perfectly fine to create a zpool backed by real devices and ordinary files.

ZFS storage for Docker on RancherOS

First, you need to stop thedocker system service and wipe out /var/lib/docker folder:

$ sudo system-docker stop docker
$ sudo rm -rf /var/lib/docker/*

To enable ZFS as the storage driver for Docker, you'll need to create a ZFS filesystem for Docker and make sure it's mounted.

$ sudo zfs create zpool1/docker
$ sudo zfs list -o name,mountpoint,mounted

At this point you'll have a ZFS filesystem created and mounted at /zpool1/docker. According to Docker ZFS storage docs, if the Docker root dir is a ZFS filesystem, the Docker daemon will automatically use zfs as its storage driver.

Now you'll need to remove -s overlay (or any other storage driver) from the Docker daemon args to allow docker to automatically detect zfs.

$ sudo ros config set rancher.docker.storage_driver ''
$ sudo ros config set rancher.docker.graph /mnt/zpool1/docker
# Now that you've changed the Docker daemon args, you'll need to start Docker
$ sudo system-docker start docker

After customizing the Docker daemon arguments and restarting docker system service, ZFS will be used as Docker storage driver:

$ docker info
Containers: 0
 Running: 0
 Paused: 0
 Stopped: 0
Images: 0
Server Version: 1.12.6
Storage Driver: zfs
 Zpool: error while getting pool information strconv.ParseUint: parsing "": invalid syntax
 Zpool Health: not available
 Parent Dataset: zpool1/docker
 Space Used By Parent: 19456
 Space Available: 8256371200
 Parent Quota: no
 Compression: off
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
 Volume: local
 Network: host bridge null overlay
Swarm: inactive
Runtimes: runc
Default Runtime: runc
Security Options: seccomp
Kernel Version: 4.9.6-rancher
Operating System: RancherOS v0.8.0-rc8
OSType: linux
Architecture: x86_64
CPUs: 1
Total Memory: 1.953 GiB
Name: ip-172-31-24-201.us-west-1.compute.internal
ID: IEE7:YTUL:Y3F5:L6LF:5WI7:LECX:YDB5:LGWZ:QRPN:4KDI:LD66:KYTC
Docker Root Dir: /mnt/zpool1/docker
Debug Mode (client): false
Debug Mode (server): false
Registry: https://index.docker.io/v1/
Insecure Registries:
 127.0.0.0/8