VDO – Virtual Data Optimizer - Compressing and Deduplicating Storage with VDO


VDO Virtual Data Optimizer - RHEL8


Red Hat Enterprise Linux 8 includes the Virtual Data Optimizer (VDO) driver, which optimizes the data footprint on block devices.

VDO is a Linux device mapper driver that reduces disk space usage on block devices, and minimizes the replication of data, saving disk space and even increasing data throughput.

VDO consists of two kernel modules and two commands:

Kernel Modules:

  • kvdo – This module to transparently control data compression.
  • uds – This module for deduplication.

Commands: 

  •  vdo- This command is used to create, remove, start, and stop VDO volumes, as well as performing other configuration changes.
  • vdostats -This command is used to report on various aspects of VDO volumes, including effective reduction and physical volume utilization. Think of this as ‘df’ for VDO capacity.

The VDO layer is placed on top of an existing block storage device, such as a RAID device or a local disk. Those block devices can also be encrypted devices. The storage layers, such as LVM logical volumes and file systems are placed on top of a VDO device.

VDO config is written in /etc/vdoconf.yml

1:10 ratio for VDO logical size Virtual machines, containers

1:3 for object storage

The following diagram shows the placement of VDO in an infrastructure consisting of KVM virtual machines that are using optimized storage devices.


VDO-based virtual machines

VDO applies three phases to data in the following order to reduce the footprint on storage devices:

1. Zero-Block Elimination filters out data blocks that contain only zeroes (0) and records the information of those blocks only in the metadata. The nonzero data blocks are then passed

to the next phase of processing. This phase enables the thin provisioning feature in the VDO devices.

2. Deduplication eliminates redundant data blocks. When you create multiple copies of the same data, VDO detects the duplicate data blocks and updates the metadata to use those

duplicate blocks as references to the original data block without creating redundant data blocks. The universal deduplication service (UDS) kernel module checks redundancy of the

data through the metadata it maintains. This kernel module ships as part of the VDO.

3. Compression is the last phase. The kvdo kernel module compresses the data blocks using LZ4 compression and groups them on 4 KB blocks.

1. Enabling VDO

Install the vdo and kmod-kvdo packages to enable VDO in the system.

[root@tower ~]# yum search vdo

yum search vdo

[root@tower2 ~]# yum install vdo kmod-kvdo

 [root@tower ~]# yum list installed | grep vdo

kmod-kvdo.x86_64                               6.2.2.117-65.el8                           @rhel-8-for-x86_64-baseos-rpms  

vdo.x86_64                                     6.2.2.117-13.el8                           @rhel-8-for-x86_64-baseos-rpms  

 If VDO services not started, then you need to start.

[root@tower ~]# systemctl status vdo.service

vdo service

2. Creating a VDO Volume

To create a VDO volume, run the vdo create command.

Make sure that you have a spare disk – or at least a partition – available for use by VDO.

[root@tower ~]# lsblk
lsblk





Next, we create the empty VDO volume on top of /dev/nvme0n2: 

[root@tower ~]# vdo create --name=vdo1 --device=/dev/nvme0n2 --vdoLogicalSize=200G

vdo creates

Verify the availability of the vdo1 volume using the vdo list command.


[root@tower ~]# vdo list

vdo1

Verify that the vdo1 volume has both the compression and deduplication features enabled.

Use grep to search for the lines containing the string Deduplication in the output of the vdo status --name=vdo1 command.

[root@tower ~]# vdo status --name=vdo1 | grep Deduplication

Deduplication: enabled

Use grep to search for the lines containing the string Compression in the output of the vdo status --name=vdo1 command.

[root@tower ~]# vdo status --name=vdo1 | grep Compression

Compression: enabled

Format the vdo1 volume with the XFS file-system type and mount it on /mnt/vdo1.

Format the vdo1 volume with the XFS file system using the mkfs command.

[root@tower ~]# mkfs.xfs -K /dev/mapper/vdo1

mkfs

The -K option in the preceding mkfs.xfs command prevents the unused blocks in the file system from being discarded immediately which lets the command return faster.

Use the udevadm command to register the new device node.

[root@tower ~]# udevadm settle

Create the /mnt/vdo1 directory using the mkdir command.

[root@tower ~]# mkdir /mnt/vdo1

Mount the vdo1 volume on /mnt/vdo1 using the mount command.

[root@tower ~]# mount /dev/mapper/vdo1 /mnt/vdo1

Permanent mount for VDO

[root@tower ~]# cat /etc/fstab | grep vdo

/dev/mapper/vdo1 /mnt/vdo1 xfs x-systemd.requires=vdo.service 0 0

[root@tower ~]# df -Th | grep vdo

/dev/mapper/vdo1 xfs   200G  1.5G  199G   1% /mnt/vdo1

View the initial statistics and status of the volume using the vdostats command.

[root@tower ~]# vdostats --hu

Device                Size  Used Available Use% Space saving%

/dev/mapper/vdo1     20.0G  4.0G 16.0G  20%       99%

To start & stop particular VDO volumes

vdo start --name=vdo1

vdo stop --name=vdo1

To activate/De-activate VDO volumes

vdo activate --name=vdo1

vdo deactivate --name=vdo1

Check status of vdo 

vdo status --name=vdo1

Enable / Disable De-duplication

vdo enableDeduplication --name=vdo1

vdo disableDeduplication --name=vdo1

Compression

vdo enableCompression --name=vdo1

vdo disableCompression --name=vdo1

To increase logical size of the VDO volume

# vdo growLogical --name=vdo1 --vdoLogicalSize=new-logical-size

To increase Physical size of the VD0 volume

# vdo growPhysical --name=vdo1

Remove VDO volumes

umount -f /dev/mapper/vdo1
udevadm settle
dmsetup remove vdo1
vdo remove --force --name=vdo1

run discard unused blocks manually

fstrim /mnt/vdo1

Conclusion

We have seen how to install and perform a basic configuration of VDO on RHEL8 / CentOS8. VDO provides native deduplication for Linux, reducing required storage capacity by eliminating redundant data from a sub-file level.

Comments

Popular posts from this blog

How to Install and Configure Kubernetes Cluster and Docker on CentOS8/RHEL8

How to install Ansible Tower on Red Hat / CentOS

How to install Ansible on RHEL8 / CentOS8