dm-clone — The Linux Kernel documentation
The Linux Kernel 7.4.0-rc3 Quick search Contents Working with the development community Development process Submitting patches Code of conduct Maintainer handbook All development-process docs Internal API manuals Core API Driver APIs Subsystems Locking Development tools and processes Licensing rules Writing documentation Development tools Testing guide Hacking guide Tracing Fault injection Livepatching Rust User-oriented documentation Administration General guides to kernel administration Booting the kernel Tracking down and identifying problems Core-kernel subsystems Block-layer and filesystem administration A block layer cache (bcache) The Android binderfs Filesystem Block Devices CIFS Device Mapper ext4 General Information File system Monitoring with fanotify NFS I/O statistics fields IBM’s Journaled File System (JFS) for Linux RAID arrays Using UFS The SGI XFS Filesystem Device-specific guides Workload analysis Everything else Build system Reporting issues Userspace tools Userspace API Firmware-related documentation Firmware Firmware and Devicetree Architecture-specific documentation CPU architectures Other documentation Unsorted documentation Translations Translations This Page Show Source dm-clone ¶ Introduction ¶ dm-clone is a device mapper target which produces a one-to-one copy of an existing, read-only source device into a writable destination device: It presents a virtual block device which makes all data appear immediately, and redirects reads and writes accordingly.
The main use case of dm-clone is to clone a potentially remote, high-latency, read-only, archival-type block device into a writable, fast, primary-type device for fast, low-latency I/O. The cloned device is visible/mountable immediately and the copy of the source device to the destination device happens in the background, in parallel with user I/O. For example, one could restore an application backup from a read-only copy, accessible through a network storage protocol (NBD, Fibre Channel, iSCSI, AoE, etc.), into a local SSD or NVMe device, and start using the device immediately, without waiting for the restore to complete.
When the cloning completes, the dm-clone table can be removed altogether and be replaced, e.g., by a linear table, mapping directly to the destination device. The dm-clone target reuses the metadata library used by the thin-provisioning target. Glossary ¶ Hydration The process of filling a region of the destination device with data from the same region of the source device, i.e., copying the region from the source to the destination device.
Once a region gets hydrated we redirect all I/O regarding it to the destination device. Design ¶ Sub-devices ¶ The target is constructed by passing three devices to it (along with other parameters detailed later): A source device - the read-only device that gets cloned and source of the hydration. A destination device - the destination of the hydration, which will become a clone of the source device. A small metadata device - it records which regions are already valid in the destination device, i.e., which regions have already been hydrated, or have been written to directly, via user I/O.
The size of the destination device must be at least equal to the size of the source device. Regions ¶ dm-clone divides the source and destination devices in fixed sized regions. Regions are the unit of hydration, i.e., the minimum amount of data copied from the source to the destination device. The region size is configurable when you first create the dm-clone device. The recommended region size is the same as the file system block size, which usually is 4KB.
The region size must be between 8 sectors (4KB) and 2097152 sectors (1GB) and a power of two. Reads and writes from/to hydrated regions are serviced from the destination device. A read to a not yet hydrated region is serviced directly from the source device. A write to a not yet hydrated region will be delayed until the corresponding region has been hydrated and the hydration of the region starts immediately. Note that a write request with size equal to region size will skip copying of the corresponding region from the source device and overwrite the region of the destination device directly.
Discards ¶ dm-clone interprets a discard request to a range that hasn’t been hydrated yet as a hint to skip hydration of the regions covered by the request, i.e., it skips copying the region’s data from the source to the destination device, and only updates its metadata. If the destination device supports discards, then by default dm-clone will pass down discard requests to it. Background Hydration ¶ dm-clone copies continuously from the source to the destination device, until all of the device has been copied.
Copying data from the source to the destination device uses bandwidth. The user can set a throttle to prevent more than a certain amount of copying occurring at any one time. Moreover, dm-clone takes into account user I/O traffic going to the devices and pauses the background hydration when there is I/O in-flight. A message hydration_threshold <#regions> can be used to set the maximum number of regions being copied, the default being 1 region. dm-clone employs dm-kcopyd for copying portions of the source device to the destination device.
By default, we issue copy requests of size equal to the region size. A message hydration_batch_size <#regions> can be used to tune the size of these copy requests. Increasing the hydration batch size results in dm-clone trying to batch together contiguous regions, so we copy the data in batches of this many regions. When the hydration of the destination device finishes, a dm event will be sent to user space. Updating on-disk metadata ¶ On-disk metadata is committed every time a FLUSH or FUA bio is written.
If no such requests are made then commits will occur every second. This means the dm-clone device behaves like a physical disk that has a volatile write cache. If power is lost you may lose some recent writes. The metadata should always be consistent in spite of any crash. Target Interface ¶ Constructor ¶ clone <metadata dev> <destination dev> <source dev> <region size> [<#feature args> [<feature arg>]* [<#core args> [<core arg>]*]] metadata dev Fast device holding the persistent metadata destination dev The destination device, where the source will be cloned source dev Read only device containing the data that gets cloned region size The size of a region in sectors #feature args Number of feature arguments passed feature args no_hydration or no_discard_passdown #core args An even number of arguments corresponding to key/value pairs passed to dm-clone core args Key/value pairs passed to dm-clone, e.g. hydration_threshold 256 Optional feature arguments are: no_hydration Create a dm-clone instance with background hydration disabled no_discard_passdown Disable passing down discards to the destination device Optional core arguments are: hydration_threshold <#regions> Maximum number of regions being copied from the source to the destination device at any one time, during background hydration. hydration_batch_size <#regions> During background hydration, try to batch together contiguous regions, so we copy data from the source to the destination device in batches of this many regions.
Status ¶ <metadata block size> <#used metadata blocks>/<#total metadata blocks> <region size> <#hydrated regions>/<#total regions> <#hydrating regions> <#feature args> <feature args>* <#core args> <core args>* <clone metadata mode> metadata block size Fixed block size for each metadata block in sectors #used metadata blocks Number of metadata blocks used #total metadata blocks Total number of metadata blocks region size Configurable region size for the device in sectors #hydrated regions Number of regions that have finished hydrating #total regions Total number of regions to hydrate #hydrating regions Number of regions currently hydrating #feature args Numbe
The rest of this page is on the live site.