From 1f4ccc82b455e2cb0cb4b24ecbaf4122a428d720 Mon Sep 17 00:00:00 2001 From: Alejandro Bonilla Date: Thu, 9 Jul 2026 20:45:30 -0400 Subject: [PATCH 01/10] docs(lvm-local-storage): add considerations for striped vs dm-thin Add a brief comparison of the two volume group types to help operators pick the right StorageClass for their workload, and append a small "Additional Notes" section covering thin-pool tuning, VM disk-bus recommendation, and Longhorn v2 coexistence. Signed-off-by: Alejandro Bonilla --- docs/advanced/addons/lvm-local-storage.md | 53 ++++++++++++++++++- .../advanced/addons/lvm-local-storage.md | 53 ++++++++++++++++++- 2 files changed, 104 insertions(+), 2 deletions(-) diff --git a/docs/advanced/addons/lvm-local-storage.md b/docs/advanced/addons/lvm-local-storage.md index 50104a3673..38d2cd6045 100644 --- a/docs/advanced/addons/lvm-local-storage.md +++ b/docs/advanced/addons/lvm-local-storage.md @@ -104,6 +104,16 @@ You can only use one type of local volume in each volume group. If necessary, cr ![](/img/v1.4/csi-driver-lvm/create-lvm-sc-05.png) +### Considerations: `striped` vs `dm-thin` + +Both volume group types are fully supported. The choice depends on how the workload will use snapshots and how the pool capacity will be shared. + +- **`striped`** is a good fit for workloads that mostly need direct volume performance across the physical devices in the volume group. Each logical volume gets its full requested capacity at provision time. Snapshots are provisioned as separate LVs sized to the origin's full capacity — a snapshot of a 100 GiB volume reserves an additional 100 GiB of volume group space at creation, regardless of how much data has actually been written to the origin. + +- **`dm-thin`** is a good fit for virtual machine workloads that take snapshots or clones, and for environments that want to over-provision pool capacity. Thin volumes consume physical space only as blocks are written to them, and snapshots are true copy-on-write at the thin-pool chunk level — a fresh snapshot of a 100 GiB volume adds effectively zero pool capacity at creation and only grows as changed blocks accumulate. + +If regular snapshots are expected, `dm-thin` is generally the right choice for VM workloads. + For more information, see [StorageClass](../storageclass.md). ## Creating a Volume with LVM @@ -177,4 +187,45 @@ You can also create a new virtual machine with the volume of the LVM StorageClas Backup creation is currently not supported. This limitation will be addressed in a future release. -::: \ No newline at end of file +::: + +## Additional Notes + +### Tuning the `dm-thin` Pool + +When the first PersistentVolumeClaim is created against a `dm-thin` StorageClass, the driver creates an LVM thin-pool named `-thinpool` at `-l 90%FREE` of the volume group. Two settings are worth knowing about: + +- **Chunk zeroing.** By default, the pool writes zeros to each newly-allocated chunk before handing it to the writer. On single-tenant clusters this can be disabled to reduce write amplification on first-touch allocations: + + ``` + sudo lvchange --zero n /-thinpool + ``` + + The change is fully reversible with `--zero y`. + +- **Pool metadata size.** LVM auto-sizes the thin-pool metadata LV at pool creation. For pools that will hold many snapshots or many thin volumes over time, extending the metadata proactively avoids running short later: + + ``` + sudo lvextend --poolmetadatasize +1G /-thinpool + ``` + +### Choosing a VM Disk Bus + +When attaching an LVM CSI PersistentVolumeClaim to a VirtualMachine, `virtio-scsi` (`bus: scsi`) generally performs better than the default `virtio-blk` (`bus: virtio`) for sustained-write workloads on thin-provisioned pools, particularly on RAID-backed storage. `virtio-scsi` supports multiple queues and has a more efficient DISCARD path. + +### Coexistence with Longhorn v2 Block-Mode Disks + +If the same node hosts a Longhorn v2 disk in block mode, the underlying device is held exclusively by the SPDK instance manager. Adding that device to the LVM `global_filter` prevents LVM's device scan from attempting to open it. Example, in `/etc/lvm/lvmlocal.conf`: + +``` +devices { + global_filter = [ + "r|/dev/loop.*|", + "r|/dev/disk/by-path/.*longhorn.*|", + "r|/dev/mapper/pvc-.*|", + "r|/dev/disk/by-id/wwn-|", + ] +} +``` + +On Harvester's immutable OS, persist the change through an `/oem/*.yaml` cloud-config file so it survives reboots. For background, see [harvester/harvester#11098](https://github.com/harvester/harvester/issues/11098). \ No newline at end of file diff --git a/versioned_docs/version-v1.8/advanced/addons/lvm-local-storage.md b/versioned_docs/version-v1.8/advanced/addons/lvm-local-storage.md index 520b2349de..49eb83ef8d 100644 --- a/versioned_docs/version-v1.8/advanced/addons/lvm-local-storage.md +++ b/versioned_docs/version-v1.8/advanced/addons/lvm-local-storage.md @@ -104,6 +104,16 @@ You can only use one type of local volume in each volume group. If necessary, cr ![](/img/v1.4/csi-driver-lvm/create-lvm-sc-05.png) +### Considerations: `striped` vs `dm-thin` + +Both volume group types are fully supported. The choice depends on how the workload will use snapshots and how the pool capacity will be shared. + +- **`striped`** is a good fit for workloads that mostly need direct volume performance across the physical devices in the volume group. Each logical volume gets its full requested capacity at provision time. Snapshots are provisioned as separate LVs sized to the origin's full capacity — a snapshot of a 100 GiB volume reserves an additional 100 GiB of volume group space at creation, regardless of how much data has actually been written to the origin. + +- **`dm-thin`** is a good fit for virtual machine workloads that take snapshots or clones, and for environments that want to over-provision pool capacity. Thin volumes consume physical space only as blocks are written to them, and snapshots are true copy-on-write at the thin-pool chunk level — a fresh snapshot of a 100 GiB volume adds effectively zero pool capacity at creation and only grows as changed blocks accumulate. + +If regular snapshots are expected, `dm-thin` is generally the right choice for VM workloads. + For more information, see [StorageClass](../storageclass.md). ## Creating a Volume with LVM @@ -177,4 +187,45 @@ You can also create a new virtual machine with the volume of the LVM StorageClas Backup creation is currently not supported. This limitation will be addressed in a future release. -::: \ No newline at end of file +::: + +## Additional Notes + +### Tuning the `dm-thin` Pool + +When the first PersistentVolumeClaim is created against a `dm-thin` StorageClass, the driver creates an LVM thin-pool named `-thinpool` at `-l 90%FREE` of the volume group. Two settings are worth knowing about: + +- **Chunk zeroing.** By default, the pool writes zeros to each newly-allocated chunk before handing it to the writer. On single-tenant clusters this can be disabled to reduce write amplification on first-touch allocations: + + ``` + sudo lvchange --zero n /-thinpool + ``` + + The change is fully reversible with `--zero y`. + +- **Pool metadata size.** LVM auto-sizes the thin-pool metadata LV at pool creation. For pools that will hold many snapshots or many thin volumes over time, extending the metadata proactively avoids running short later: + + ``` + sudo lvextend --poolmetadatasize +1G /-thinpool + ``` + +### Choosing a VM Disk Bus + +When attaching an LVM CSI PersistentVolumeClaim to a VirtualMachine, `virtio-scsi` (`bus: scsi`) generally performs better than the default `virtio-blk` (`bus: virtio`) for sustained-write workloads on thin-provisioned pools, particularly on RAID-backed storage. `virtio-scsi` supports multiple queues and has a more efficient DISCARD path. + +### Coexistence with Longhorn v2 Block-Mode Disks + +If the same node hosts a Longhorn v2 disk in block mode, the underlying device is held exclusively by the SPDK instance manager. Adding that device to the LVM `global_filter` prevents LVM's device scan from attempting to open it. Example, in `/etc/lvm/lvmlocal.conf`: + +``` +devices { + global_filter = [ + "r|/dev/loop.*|", + "r|/dev/disk/by-path/.*longhorn.*|", + "r|/dev/mapper/pvc-.*|", + "r|/dev/disk/by-id/wwn-|", + ] +} +``` + +On Harvester's immutable OS, persist the change through an `/oem/*.yaml` cloud-config file so it survives reboots. For background, see [harvester/harvester#11098](https://github.com/harvester/harvester/issues/11098). \ No newline at end of file From 4d047a82f48a818a395f4c2799c3b9f3ac13787e Mon Sep 17 00:00:00 2001 From: Alejandro Bonilla Date: Mon, 13 Jul 2026 10:27:01 -0400 Subject: [PATCH 02/10] Update docs/advanced/addons/lvm-local-storage.md Co-authored-by: Jillian Maroket <67180770+jillian-maroket@users.noreply.github.com> Signed-off-by: Alejandro Bonilla --- docs/advanced/addons/lvm-local-storage.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/advanced/addons/lvm-local-storage.md b/docs/advanced/addons/lvm-local-storage.md index 38d2cd6045..8a7c671b46 100644 --- a/docs/advanced/addons/lvm-local-storage.md +++ b/docs/advanced/addons/lvm-local-storage.md @@ -228,4 +228,4 @@ devices { } ``` -On Harvester's immutable OS, persist the change through an `/oem/*.yaml` cloud-config file so it survives reboots. For background, see [harvester/harvester#11098](https://github.com/harvester/harvester/issues/11098). \ No newline at end of file +Because Harvester's operating system is immutable, you must persist this change through an `/oem/*.yaml` cloud-config file to ensure it survives system reboots and upgrades.For more information, see issue [#11098](https://github.com/harvester/harvester/issues/11098). \ No newline at end of file From f2730fb3f4e31ae496a51fb9dd0254b700d77dfa Mon Sep 17 00:00:00 2001 From: Alejandro Bonilla Date: Mon, 13 Jul 2026 10:30:11 -0400 Subject: [PATCH 03/10] Update docs/advanced/addons/lvm-local-storage.md Co-authored-by: Jillian Maroket <67180770+jillian-maroket@users.noreply.github.com> Signed-off-by: Alejandro Bonilla --- docs/advanced/addons/lvm-local-storage.md | 14 +++++++++++++- 1 file changed, 13 insertions(+), 1 deletion(-) diff --git a/docs/advanced/addons/lvm-local-storage.md b/docs/advanced/addons/lvm-local-storage.md index 8a7c671b46..c7c321656c 100644 --- a/docs/advanced/addons/lvm-local-storage.md +++ b/docs/advanced/addons/lvm-local-storage.md @@ -98,7 +98,19 @@ You can only use one type of local volume in each volume group. If necessary, cr ![](/img/v1.4/csi-driver-lvm/create-lvm-sc-04.png) -1. Click **Save**. + - **Volume Group Type**: Select a type based on how the workload uses snapshots and how pool capacity is allocated. Harvester supports the following options: + + - **striped**: Best suited for workloads requiring direct, high-performance volume access distributed across the physical devices in the volume group. Each logical volume is fully allocated its requested capacity at provisioning time. Snapshots are created as independent logical volumes sized to match the source volume's maximum capacity. For example, a snapshot of a 100 GiB volume reserves an additional 100 GiB of volume group space upon creation, regardless of the actual quantity of data written to the source.. + + - **dm-thin**: Best suited for virtual machine workloads that frequently use snapshots or clones, and for environments that require over-provisioned pool capacity. Thin-provisioned volumes consume physical space only as blocks are written. Snapshots leverage true copy-on-write functionality at the thin-pool chunk level. For example, a fresh snapshot of a 100 GiB volume consumes practically zero pool capacity at creation, only growing as modified blocks accumulate over time. + + :::tip + + Select **dm-thin** if you expect to create regular snapshots or scheduled backups. This option is generally optimal for standard virtual machine workloads because of its efficient space utilization. + + ::: + + ![](/img/v1.4/csi-driver-lvm/create-lvm-sc-04.png) 1. On the **Storage** screen, verify that the StorageClass was created and the correct provisioner was set. From 959d31bde32248c2e80491a4900e25c7a5470ac0 Mon Sep 17 00:00:00 2001 From: Alejandro Bonilla Date: Mon, 13 Jul 2026 10:30:19 -0400 Subject: [PATCH 04/10] Update docs/advanced/addons/lvm-local-storage.md Co-authored-by: Jillian Maroket <67180770+jillian-maroket@users.noreply.github.com> Signed-off-by: Alejandro Bonilla --- docs/advanced/addons/lvm-local-storage.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/advanced/addons/lvm-local-storage.md b/docs/advanced/addons/lvm-local-storage.md index c7c321656c..2792af5fcd 100644 --- a/docs/advanced/addons/lvm-local-storage.md +++ b/docs/advanced/addons/lvm-local-storage.md @@ -213,9 +213,9 @@ When the first PersistentVolumeClaim is created against a `dm-thin` StorageClass sudo lvchange --zero n /-thinpool ``` - The change is fully reversible with `--zero y`. + You can fully reverse the change by running the command with `--zero y`. -- **Pool metadata size.** LVM auto-sizes the thin-pool metadata LV at pool creation. For pools that will hold many snapshots or many thin volumes over time, extending the metadata proactively avoids running short later: +- **Pool metadata size**: When the thin pool is created, LVM automatically sizes its metadata logical volume. Consider extending this volume proactively if you expect the pool to store a large number of snapshots or thin volumes over time. Doing so prevents the pool from running out of space and becoming unresponsive later. ``` sudo lvextend --poolmetadatasize +1G /-thinpool From c94296d2d08cffc159a862078226a4f6370a784f Mon Sep 17 00:00:00 2001 From: Alejandro Bonilla Date: Mon, 13 Jul 2026 10:30:24 -0400 Subject: [PATCH 05/10] Update docs/advanced/addons/lvm-local-storage.md Co-authored-by: Jillian Maroket <67180770+jillian-maroket@users.noreply.github.com> Signed-off-by: Alejandro Bonilla --- docs/advanced/addons/lvm-local-storage.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/advanced/addons/lvm-local-storage.md b/docs/advanced/addons/lvm-local-storage.md index 2792af5fcd..8022711a1e 100644 --- a/docs/advanced/addons/lvm-local-storage.md +++ b/docs/advanced/addons/lvm-local-storage.md @@ -207,7 +207,7 @@ Backup creation is currently not supported. This limitation will be addressed in When the first PersistentVolumeClaim is created against a `dm-thin` StorageClass, the driver creates an LVM thin-pool named `-thinpool` at `-l 90%FREE` of the volume group. Two settings are worth knowing about: -- **Chunk zeroing.** By default, the pool writes zeros to each newly-allocated chunk before handing it to the writer. On single-tenant clusters this can be disabled to reduce write amplification on first-touch allocations: +- **Chunk zeroing**: By default, the thin pool writes zeros to each newly allocated block chunk before exposing it to a write operation. On single-tenant clusters, you can disable chunk zeroing to significantly reduce write amplification during initial data allocations. ``` sudo lvchange --zero n /-thinpool From 3ac3f634ded502248e8b3cf5ad9bd25c5e53d6ca Mon Sep 17 00:00:00 2001 From: Alejandro Bonilla Date: Mon, 13 Jul 2026 10:30:30 -0400 Subject: [PATCH 06/10] Update docs/advanced/addons/lvm-local-storage.md Co-authored-by: Jillian Maroket <67180770+jillian-maroket@users.noreply.github.com> Signed-off-by: Alejandro Bonilla --- docs/advanced/addons/lvm-local-storage.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/advanced/addons/lvm-local-storage.md b/docs/advanced/addons/lvm-local-storage.md index 8022711a1e..5a0a32c5e1 100644 --- a/docs/advanced/addons/lvm-local-storage.md +++ b/docs/advanced/addons/lvm-local-storage.md @@ -205,7 +205,7 @@ Backup creation is currently not supported. This limitation will be addressed in ### Tuning the `dm-thin` Pool -When the first PersistentVolumeClaim is created against a `dm-thin` StorageClass, the driver creates an LVM thin-pool named `-thinpool` at `-l 90%FREE` of the volume group. Two settings are worth knowing about: +When the first PersistentVolumeClaim is created against a `dm-thin` StorageClass, the driver creates an LVM thin pool named `-thinpool` using `-l 90%FREE` (allocating 90% of the volume group's remaining free space). Consider tuning the following settings based on your workload demands: - **Chunk zeroing**: By default, the thin pool writes zeros to each newly allocated block chunk before exposing it to a write operation. On single-tenant clusters, you can disable chunk zeroing to significantly reduce write amplification during initial data allocations. From 2e8b93c0bcf0883524aecc4f821c30bcbfe71eaf Mon Sep 17 00:00:00 2001 From: Alejandro Bonilla Date: Mon, 13 Jul 2026 10:30:43 -0400 Subject: [PATCH 07/10] Update docs/advanced/addons/lvm-local-storage.md Co-authored-by: Jillian Maroket <67180770+jillian-maroket@users.noreply.github.com> Signed-off-by: Alejandro Bonilla --- docs/advanced/addons/lvm-local-storage.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/advanced/addons/lvm-local-storage.md b/docs/advanced/addons/lvm-local-storage.md index 5a0a32c5e1..f1db7bcc49 100644 --- a/docs/advanced/addons/lvm-local-storage.md +++ b/docs/advanced/addons/lvm-local-storage.md @@ -221,9 +221,9 @@ When the first PersistentVolumeClaim is created against a `dm-thin` StorageClass sudo lvextend --poolmetadatasize +1G /-thinpool ``` -### Choosing a VM Disk Bus +### Choosing a Virtual Machine Disk Bus -When attaching an LVM CSI PersistentVolumeClaim to a VirtualMachine, `virtio-scsi` (`bus: scsi`) generally performs better than the default `virtio-blk` (`bus: virtio`) for sustained-write workloads on thin-provisioned pools, particularly on RAID-backed storage. `virtio-scsi` supports multiple queues and has a more efficient DISCARD path. +When attaching an LVM CSI PersistentVolumeClaim to a VirtualMachine, `virtio-scsi` (`bus: scsi`) generally outperforms the default `virtio-blk` (`bus: virtio`) for sustained-write workloads on thin-provisioned pools, particularly on RAID-backed storage. `virtio-scsi` performs better because it supports multiple queues and uses a more efficient DISCARD path. ### Coexistence with Longhorn v2 Block-Mode Disks From 3ca804ed17526540def358b07dc0e4fb159ea358 Mon Sep 17 00:00:00 2001 From: Alejandro Bonilla Date: Mon, 13 Jul 2026 10:30:50 -0400 Subject: [PATCH 08/10] Update docs/advanced/addons/lvm-local-storage.md Co-authored-by: Jillian Maroket <67180770+jillian-maroket@users.noreply.github.com> Signed-off-by: Alejandro Bonilla --- docs/advanced/addons/lvm-local-storage.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/advanced/addons/lvm-local-storage.md b/docs/advanced/addons/lvm-local-storage.md index f1db7bcc49..8b356c42e1 100644 --- a/docs/advanced/addons/lvm-local-storage.md +++ b/docs/advanced/addons/lvm-local-storage.md @@ -227,7 +227,7 @@ When attaching an LVM CSI PersistentVolumeClaim to a VirtualMachine, `virtio-scs ### Coexistence with Longhorn v2 Block-Mode Disks -If the same node hosts a Longhorn v2 disk in block mode, the underlying device is held exclusively by the SPDK instance manager. Adding that device to the LVM `global_filter` prevents LVM's device scan from attempting to open it. Example, in `/etc/lvm/lvmlocal.conf`: +If the same node hosts a Longhorn V2 disk in block mode, the underlying device is held exclusively by the SPDK Instance Manager. You can add this device to the LVM `global_filter` to exclude it from LVM device scans and prevent resource conflicts. ``` devices { From c36d7d71efdfdab91e69ed5b9ed9d1c5612a5218 Mon Sep 17 00:00:00 2001 From: Alejandro Bonilla Date: Mon, 13 Jul 2026 10:35:22 -0400 Subject: [PATCH 09/10] docs(lvm-local-storage): remove redundant "Considerations" section Jillian moved the striped vs dm-thin explanation into the procedure step above (Volume Group Type), so the standalone section is now duplicate content. The "For more information, see [StorageClass]" link is kept. Co-authored-by: Jillian Maroket <67180770+jillian-maroket@users.noreply.github.com> Signed-off-by: Alejandro Bonilla --- docs/advanced/addons/lvm-local-storage.md | 10 ---------- 1 file changed, 10 deletions(-) diff --git a/docs/advanced/addons/lvm-local-storage.md b/docs/advanced/addons/lvm-local-storage.md index 8b356c42e1..5a53a486e7 100644 --- a/docs/advanced/addons/lvm-local-storage.md +++ b/docs/advanced/addons/lvm-local-storage.md @@ -116,16 +116,6 @@ You can only use one type of local volume in each volume group. If necessary, cr ![](/img/v1.4/csi-driver-lvm/create-lvm-sc-05.png) -### Considerations: `striped` vs `dm-thin` - -Both volume group types are fully supported. The choice depends on how the workload will use snapshots and how the pool capacity will be shared. - -- **`striped`** is a good fit for workloads that mostly need direct volume performance across the physical devices in the volume group. Each logical volume gets its full requested capacity at provision time. Snapshots are provisioned as separate LVs sized to the origin's full capacity — a snapshot of a 100 GiB volume reserves an additional 100 GiB of volume group space at creation, regardless of how much data has actually been written to the origin. - -- **`dm-thin`** is a good fit for virtual machine workloads that take snapshots or clones, and for environments that want to over-provision pool capacity. Thin volumes consume physical space only as blocks are written to them, and snapshots are true copy-on-write at the thin-pool chunk level — a fresh snapshot of a 100 GiB volume adds effectively zero pool capacity at creation and only grows as changed blocks accumulate. - -If regular snapshots are expected, `dm-thin` is generally the right choice for VM workloads. - For more information, see [StorageClass](../storageclass.md). ## Creating a Volume with LVM From 72d8748452dc59e8268849d35cc9bc03b7a07006 Mon Sep 17 00:00:00 2001 From: Alejandro Bonilla Date: Tue, 14 Jul 2026 08:41:36 -0400 Subject: [PATCH 10/10] docs(lvm-local-storage): document chunk size + expand tuning section Field investigation showed that Harvester's LVM CSI driver, when creating a new thin pool, was letting LVM auto-select the chunk_size, which for multi-TB pools produces 8-16 MiB chunks. On random 4K workloads this causes severe write amplification (up to 4096:1). A field customer measured DI_RANDOM = 326 IOPS on hardware capable of 10x that, traced to a 16 MiB chunk size. harvester/csi-driver-lvm PR #57 (still in review) exposes three new StorageClass parameters (chunkSize, poolMetadataSize, zeroBlocks) and ships sensible defaults (512K/16G/false). This docs change explains the choice, when to override, the hard "cannot change after pool creation" constraint, and gives a full example StorageClass with all three parameters set. Also expands the existing chunk-zeroing and pool-metadata subsections to reference the new StorageClass parameters (in addition to the post-creation lvchange/lvextend workflows they already documented). Only edits the current-dev docs/ tree - not backported to versioned_docs/, since the StorageClass parameters ship in a CSI driver version that will target Harvester v1.9. The post-creation lvchange/lvextend commands work on older Harvester versions, but the chunkSize parameter itself does not. Base is docs-lvm-considerations (PR #1075), which introduces the initial "Tuning the dm-thin Pool" section. Rebasing onto main once #1075 merges. Signed-off-by: Alejandro Bonilla --- docs/advanced/addons/lvm-local-storage.md | 48 ++++++++++++++++++++++- 1 file changed, 47 insertions(+), 1 deletion(-) diff --git a/docs/advanced/addons/lvm-local-storage.md b/docs/advanced/addons/lvm-local-storage.md index 5a53a486e7..862107e7c6 100644 --- a/docs/advanced/addons/lvm-local-storage.md +++ b/docs/advanced/addons/lvm-local-storage.md @@ -197,20 +197,66 @@ Backup creation is currently not supported. This limitation will be addressed in When the first PersistentVolumeClaim is created against a `dm-thin` StorageClass, the driver creates an LVM thin pool named `-thinpool` using `-l 90%FREE` (allocating 90% of the volume group's remaining free space). Consider tuning the following settings based on your workload demands: +- **Chunk size**: The chunk size determines the smallest unit of physical space that a thin pool allocates in response to a write. A write to a previously-unallocated region always allocates a full chunk, so small random writes to virgin space against a large chunk size cause severe write amplification (a 4 KiB write against a 16 MiB chunk allocates 16 MiB of pool space). When you install the harvester-csi-driver-lvm add-on version 0.4.0 or later, the default StorageClass sets `chunkSize: "512K"`, which matches [LVM's upstream `performance` policy seed](https://github.com/lvmteam/lvm2/blob/master/lib/config/defaults.h) and the [Linux kernel dm-thin admin guide's general-purpose recommendation](https://docs.kernel.org/admin-guide/device-mapper/thin-provisioning.html), and comfortably supports thin pools up to 128 TB. + + :::warning + + Chunk size cannot be changed after the thin pool is created. The value must be chosen before the first PersistentVolumeClaim is created against the StorageClass. Reducing chunk size on an existing pool requires evacuating all volumes, destroying the pool, recreating it with the new chunk size, and restoring the volumes. + + ::: + + Override the default with the `chunkSize` parameter on the StorageClass. Common values: + + | Value | When to use | + |---|---| + | `128K` | Snapshot-heavy tenants (halves copy-on-write cost per touched region) | + | `512K` | **Default** — general-purpose VM workloads on pools up to 128 TB | + | `1M` | Pools larger than 128 TB, or bulk sequential-write workloads | + + Never exceed `2M`. Larger chunks trigger disproportionate copy-on-write costs for snapshots, as documented in the [Red Hat Gluster admin guide](https://docs.redhat.com/en/documentation/red_hat_gluster_storage/3.5/html/administration_guide/chap-configuring_red_hat_storage_for_enhancing_performance). If you rely on LVM's built-in auto-selection instead of setting `chunkSize` explicitly, LVM chooses the chunk size based on the pool size to keep metadata bounded, which for multi-TB pools produces 8–16 MiB chunks — appropriate for metadata sizing but often not for random-write performance. + + Verify the effective chunk size of a live pool: + + ``` + sudo lvs -o vg_name,lv_name,chunk_size,zero /-thinpool + sudo dmsetup table ---thinpool-tpool + ``` + - **Chunk zeroing**: By default, the thin pool writes zeros to each newly allocated block chunk before exposing it to a write operation. On single-tenant clusters, you can disable chunk zeroing to significantly reduce write amplification during initial data allocations. + Set `zeroBlocks: "false"` on the StorageClass at pool creation time, or apply it to an existing pool with: + ``` sudo lvchange --zero n /-thinpool ``` You can fully reverse the change by running the command with `--zero y`. -- **Pool metadata size**: When the thin pool is created, LVM automatically sizes its metadata logical volume. Consider extending this volume proactively if you expect the pool to store a large number of snapshots or thin volumes over time. Doing so prevents the pool from running out of space and becoming unresponsive later. +- **Pool metadata size**: When the thin pool is created, the driver sizes its metadata logical volume based on the `poolMetadataSize` StorageClass parameter (default `16G`, sufficient for pools up to 128 TB at the default 512 KiB chunk size). If you set `chunkSize` smaller than 512 KiB, the pool needs more metadata to address the same amount of data: metadata bytes ≈ 64 × (pool_size ÷ chunk_size). If you did not set the parameter, or if the pool grew significantly after creation, extend the metadata volume proactively to prevent the pool from running out of space and becoming unresponsive later. ``` sudo lvextend --poolmetadatasize +1G /-thinpool ``` +- **Example `dm-thin` StorageClass with all tuning parameters set explicitly**: + + ```yaml + apiVersion: storage.k8s.io/v1 + kind: StorageClass + metadata: + name: harvester-lvm-thin + provisioner: lvm.driver.harvesterhci.io + parameters: + type: dm-thin + vgName: vmvg + chunkSize: "512K" # Applied on first-time pool creation only. + poolMetadataSize: "16G" # Applied on first-time pool creation only. + zeroBlocks: "false" # Applied on first-time pool creation only. + reclaimPolicy: Delete + volumeBindingMode: WaitForFirstConsumer + allowVolumeExpansion: true + ``` + ### Choosing a Virtual Machine Disk Bus When attaching an LVM CSI PersistentVolumeClaim to a VirtualMachine, `virtio-scsi` (`bus: scsi`) generally outperforms the default `virtio-blk` (`bus: virtio`) for sustained-write workloads on thin-provisioned pools, particularly on RAID-backed storage. `virtio-scsi` performs better because it supports multiple queues and uses a more efficient DISCARD path.