VDO – Virtual Data Optimizer - Compressing and Deduplicating Storage with VDO
Red Hat Enterprise Linux 8 includes the
Virtual Data Optimizer (VDO) driver, which optimizes the data footprint on
block devices.
VDO is a Linux device mapper driver that
reduces disk space usage on block devices, and minimizes the replication of
data, saving disk space and even increasing data throughput.
VDO consists of two kernel modules and two
commands:
Kernel Modules:
- kvdo – This module to transparently control data compression.
- uds – This module for deduplication.
Commands:
- vdo- This command is used to create, remove, start, and stop VDO volumes, as well as performing other configuration changes.
- vdostats -This command is used to report on various aspects of VDO volumes, including effective reduction and physical volume utilization. Think of this as ‘df’ for VDO capacity.
The VDO layer is placed on top of an
existing block storage device, such as a RAID device or a local disk. Those
block devices can also be encrypted devices. The storage layers, such as LVM
logical volumes and file systems are placed on top of a VDO device.
VDO config is written in /etc/vdoconf.yml
1:10 ratio for VDO logical size Virtual
machines, containers
1:3 for object storage
The following diagram shows the placement
of VDO in an infrastructure consisting of KVM virtual machines that are using
optimized storage devices.
VDO applies three phases to data in the
following order to reduce the footprint on storage devices:
1. Zero-Block Elimination filters
out data blocks that contain only zeroes (0) and records the information of
those blocks only in the metadata. The nonzero data blocks are then passed
to the next phase of processing. This
phase enables the thin provisioning feature in the VDO devices.
2. Deduplication eliminates
redundant data blocks. When you create multiple copies of the same data, VDO
detects the duplicate data blocks and updates the metadata to use those
duplicate blocks as references to the
original data block without creating redundant data blocks. The universal
deduplication service (UDS) kernel module checks redundancy of the
data through the metadata it maintains.
This kernel module ships as part of the VDO.
3. Compression is the last phase.
The kvdo kernel module compresses the data blocks using LZ4 compression and
groups them on 4 KB blocks.
1. Enabling VDO
Install the vdo and kmod-kvdo packages to enable VDO in the system.
If VDO services not started, then you need to start.
2. Creating a VDO Volume
To create a VDO volume, run the vdo
create command.
Make sure that you have a spare disk – or
at least a partition – available for use by VDO.
Next, we create the empty VDO volume on top of /dev/nvme0n2:
Verify the availability of the vdo1 volume
using the vdo list command.
Verify that the vdo1 volume has both the
compression and deduplication features enabled.
Use grep to search for the lines containing the string Deduplication in the output of the vdo status --name=vdo1 command.
Use grep to search for the lines containing the string Compression in the output of the vdo status --name=vdo1 command.
Format the vdo1 volume with the XFS
file-system type and mount it on /mnt/vdo1.
Format the vdo1 volume with the XFS file system using the mkfs command.
The -K option in the preceding mkfs.xfs
command prevents the unused blocks in the file system from being discarded
immediately which lets the command return faster.
Use the udevadm command to register the new device node.
Create the /mnt/vdo1 directory using the mkdir command.
Mount the vdo1 volume on /mnt/vdo1 using the mount command.
Permanent mount for VDO
View the initial statistics and status of the volume using the vdostats command.
To start & stop particular VDO volumes
To activate/De-activate VDO volumes
Check status of vdo Enable / Disable De-duplication
Compression
To increase logical size of the VDO volume
To increase Physical size of the VD0 volume
Remove VDO volumes
umount -f /dev/mapper/vdo1
udevadm settle
dmsetup remove vdo1
vdo remove --force --name=vdo1
run discard unused blocks manually
Comments
Post a Comment