VDO – Virtual Data Optimizer - Compressing and Deduplicating Storage with VDO

Red Hat Enterprise Linux 8 includes the Virtual Data Optimizer (VDO) driver, which optimizes the data footprint on block devices.

VDO is a Linux device mapper driver that reduces disk space usage on block devices, and minimizes the replication of data, saving disk space and even increasing data throughput.

VDO consists of two kernel modules and two commands:

Kernel Modules:

• kvdo – This module to transparently control data compression.
• uds – This module for deduplication.

Commands: 

•  vdo- This command is used to create, remove, start, and stop VDO volumes, as well as performing other configuration changes.
• vdostats -This command is used to report on various aspects of VDO volumes, including effective reduction and physical volume utilization. Think of this as ‘df’ for VDO capacity.

The VDO layer is placed on top of an existing block storage device, such as a RAID device or a local disk. Those block devices can also be encrypted devices. The storage layers, such as LVM logical volumes and file systems are placed on top of a VDO device.

VDO config is written in /etc/vdoconf.yml

1:10 ratio for VDO logical size Virtual machines, containers

1:3 for object storage

The following diagram shows the placement of VDO in an infrastructure consisting of KVM virtual machines that are using optimized storage devices.

VDO applies three phases to data in the following order to reduce the footprint on storage devices:

1. Zero-Block Elimination filters out data blocks that contain only zeroes (0) and records the information of those blocks only in the metadata. The nonzero data blocks are then passed

to the next phase of processing. This phase enables the thin provisioning feature in the VDO devices.

2. Deduplication eliminates redundant data blocks. When you create multiple copies of the same data, VDO detects the duplicate data blocks and updates the metadata to use those

duplicate blocks as references to the original data block without creating redundant data blocks. The universal deduplication service (UDS) kernel module checks redundancy of the

data through the metadata it maintains. This kernel module ships as part of the VDO.

3. Compression is the last phase. The kvdo kernel module compresses the data blocks using LZ4 compression and groups them on 4 KB blocks.

1. Enabling VDO

Install the vdo and kmod-kvdo packages to enable VDO in the system.

[root@tower ~]# yum search vdo

[root@tower2 ~]# yum install vdo kmod-kvdo
 [root@tower ~]# yum list installed | grep vdo
kmod-kvdo.x86_64                               	6.2.2.117-65.el8                           	@rhel-8-for-x86_64-baseos-rpms  
vdo.x86_64                                     	6.2.2.117-13.el8                           	@rhel-8-for-x86_64-baseos-rpms  

 If VDO services not started, then you need to start.

[root@tower ~]# systemctl status vdo.service

2. Creating a VDO Volume

To create a VDO volume, run the vdo create command.

Make sure that you have a spare disk – or at least a partition – available for use by VDO.

[root@tower ~]# lsblk

Next, we create the empty VDO volume on top of /dev/nvme0n2: 

[root@tower ~]# vdo create --name=vdo1 --device=/dev/nvme0n2 --vdoLogicalSize=200G

Verify the availability of the vdo1 volume using the vdo list command.

[root@tower ~]# vdo list
vdo1

Verify that the vdo1 volume has both the compression and deduplication features enabled.

Use grep to search for the lines containing the string Deduplication in the output of the vdo status --name=vdo1 command.

[root@tower ~]# vdo status --name=vdo1 | grep Deduplication
	Deduplication: enabled

Use grep to search for the lines containing the string Compression in the output of the vdo status --name=vdo1 command.

[root@tower ~]# vdo status --name=vdo1 | grep Compression
	Compression: enabled

Format the vdo1 volume with the XFS file-system type and mount it on /mnt/vdo1.

Format the vdo1 volume with the XFS file system using the mkfs command.

[root@tower ~]# mkfs.xfs -K /dev/mapper/vdo1

The -K option in the preceding mkfs.xfs command prevents the unused blocks in the file system from being discarded immediately which lets the command return faster.

Use the udevadm command to register the new device node.

[root@tower ~]# udevadm settle

Create the /mnt/vdo1 directory using the mkdir command.

[root@tower ~]# mkdir /mnt/vdo1

Mount the vdo1 volume on /mnt/vdo1 using the mount command.

[root@tower ~]# mount /dev/mapper/vdo1 /mnt/vdo1

Permanent mount for VDO

[root@tower ~]# cat /etc/fstab | grep vdo
/dev/mapper/vdo1 /mnt/vdo1 xfs x-systemd.requires=vdo.service 0 0
[root@tower ~]# df -Th | grep vdo
/dev/mapper/vdo1 xfs   	200G  1.5G  199G   1% /mnt/vdo1

View the initial statistics and status of the volume using the vdostats command.

[root@tower ~]# vdostats --hu
Device                	Size  	Used Available Use% Space saving%
/dev/mapper/vdo1     	20.0G  	4.0G 	16.0G  20%       	99%

To start & stop particular VDO volumes

vdo start --name=vdo1
vdo stop --name=vdo1

To activate/De-activate VDO volumes

vdo activate --name=vdo1
vdo deactivate --name=vdo1

Check status of vdo 

vdo status --name=vdo1

Enable / Disable De-duplication

vdo enableDeduplication --name=vdo1
vdo disableDeduplication --name=vdo1

Compression

vdo enableCompression --name=vdo1
vdo disableCompression --name=vdo1

To increase logical size of the VDO volume

# vdo growLogical --name=vdo1 --vdoLogicalSize=new-logical-size

To increase Physical size of the VD0 volume

# vdo growPhysical --name=vdo1

Remove VDO volumes

umount -f /dev/mapper/vdo1
udevadm settle
dmsetup remove vdo1
vdo remove --force --name=vdo1

run discard unused blocks manually

fstrim /mnt/vdo1

Conclusion

We have seen how to install and perform a basic configuration of VDO on RHEL8 / CentOS8. VDO provides native deduplication for Linux, reducing required storage capacity by eliminating redundant data from a sub-file level.