Skip to content

Commit 2194fed

Browse files
v-tappelgatev-tappelgate
committed
Checklist merge/revisions
1 parent 9b0a8e4 commit 2194fed

1 file changed

Lines changed: 120 additions & 38 deletions

File tree

‎support/windows-server/backup-and-storage/windows-server-mpio-troubleshooting.md‎

Lines changed: 120 additions & 38 deletions
Original file line numberDiff line numberDiff line change
@@ -21,13 +21,6 @@ MPIO or disk storage failures cause several problems. You see disk unavailabilit
2121

2222
## Known issues
2323

24-
### Windows Server 2019 or 2022: VM IO performance degrades when Hyper-V uses resilient change tracking (RCT) is in use
25-
26-
If you experience this issue, make sure that your computers are up to date. [October 23, 2025—KB5070884 (OS Build 20348.4297)](https://support.microsoft.com/en-us/topic/october-23-2025-kb5070884-os-build-20348-4297-out-of-band-9c001fdc-f0d2-4636-87bb-494a59da55d0) and subsequent updates contain a fix for this issue.
27-
28-
> [!NOTE]
29-
> After you install this update, VMs that have been backed up by using or a host-level backup application might not be able to start. To fix this issue, delete any .rct and .mrt files that are associated with the affected virtual hard disks. Then try again to start the VMs. If the issue persists, contact Microsoft Support.
30-
3124
### After you update firmware, multipath disks are missing
3225

3326
This is a known issue for some storage devices. Contact your storage vendor for support.
@@ -36,19 +29,6 @@ This is a known issue for some storage devices. Contact your storage vendor for
3629

3730
This behavior reflects a Storport driver issue. Make sure that your drivers are up to date. If the issue continues, contact your vendor for support.
3831

39-
### Some cluster disks remain in "Online Pending" state
40-
41-
The default `PendingTimeout` value is too low for cluster disks that have quota or File Server Resource Manager (FSRM) changes. As a result, the cluster disks don't come online. To increase this `PendingTimout` value, run the following cmdlets at the PowerShell command prompt:
42-
43-
```powershell
44-
Get-ClusterResource "<Resource_Name>" | Set-ClusterParameter PendingTimeout 300000
45-
```
46-
47-
> [!NOTE]
48-
>
49-
> - In these commands, \<Resource_Name> is the name of the disk resource.
50-
> - The value of the PendingTimeout property is measured in milliseconds. The value shown here is higher than the default value.
51-
5232
## Troubleshooting checklist
5333

5434
Use this checklist for systematic troubleshooting.
@@ -79,18 +59,133 @@ Use this checklist for systematic troubleshooting.
7959
- Did any servers restart recently?
8060
- Are the relevant Windows features, such as MPIO, installed? Are the related services running?
8161

82-
### Investigate issues more deeply
62+
### Verify the MPIO and storage configuration
63+
64+
1. To review the device IDs of the storage hardware, follow these steps:
65+
1. In the search bar, type *mpiocpl*, and then in the search results, select **MPIO**.
66+
1. Select **Discover Multi-Paths**. Review the list of IDs and fix any errors.
67+
> [!NOTE]
68+
> You can also use the Windows PowerShell `Get-MPIOAvailableHW` and `Get-MPIOSetting` cmdlets, or the command line `mpclaim -e` and `mpclaim -r -i -a` commands to perform these steps.
69+
70+
1. To review and update the storage area network (SAN) policy, open a Windows command prompt window and then run the following command:
71+
**Review and update SAN policy**:
72+
73+
```console
74+
diskpart san policy=OnlineAll
75+
```
76+
77+
1. To update the path configuration, follow these guidelines:
78+
1. For redundancy, make sure each iSCSI connection uses a different network adapter.
79+
1. If the system only detects one path, rebuild the iSCSI connections.
80+
81+
### Update drivers and firmware
82+
83+
- Update all storage, network, and multipath drivers to the latest supported versions from your hardware or storage vendor.
84+
- Update storage or SAN controller firmware as recommended.
85+
86+
### Verify the health of the file system and disks
87+
88+
- To check and repair file system errors, run `chkdsk <drive>: /f /r` at a Windows command prompt.
89+
- If chkdsk fails while scanning a RAW volume, attempt to recover the data or restore it from a backup.
90+
- If the resilient file system (ReFS) is unstable, consider reformatting the file system to NTFS.
91+
92+
### Check for cluster and CSV disk issues
93+
94+
- To check the status of the disk resources, Use Failover Cluster Manager.
95+
- If your disk resource is stuck in the "Online Pending" state, or if the disk resource can't come online and generates error 1069 or error 1205, the timeout setting might be too short. To increase this value, run the following cmdlets at the PowerShell command prompt:
96+
97+
```powershell
98+
Get-ClusterResource "<Resource_Name>" | Set-ClusterParameter PendingTimeout 300000
99+
```
100+
101+
> [!NOTE]
102+
>
103+
> - In these commands, \<Resource_Name> is the name of the disk resource.
104+
> - The value of the PendingTimeout property is measured in milliseconds. The value shown here is higher than the default value.
105+
106+
- For stale ClusterStorage folders, stop cluster service, take ownership and permissions with `takeown` and `icacls`, then delete with `rmdir`.
107+
108+
### Check the state of the MPIO paths
109+
110+
- If only some paths are visible, verify zoning, cabling, and HBA status on both server and the SAN.
111+
- If you observe Event IDs 153, 129, or 140, you might have storage or network bottlenecks. Review your underlying storage or network traffic.
112+
- If MPIO paths are missing or incorrectly configured following a restart, see [After maintenance or a restart, administrative tools don't show disks or paths](#after-maintenance-or-a-restart-administrative-tools-dont-show-disks-or-paths) for instructions.
113+
114+
### Tune Registry and policy settings
115+
116+
[!INCLUDE [Registry important alert](../../../includes/registry-important-alert.md)]
117+
118+
- If you observe excessive logging (such as unusual growth of the lsass.log file), set the following registry value.
119+
120+
- Subkey: `HKLM\SYSTEM\CurrentControlSet\Control\Lsa`
121+
- Value, data, and type: `LogToFile` = `0` (DWORD)
122+
After you change a registry value, restart the computer.
123+
124+
- To configure MPIO verification settings, run the following cmdlets at a PowerShell command prompt window:
83125

84-
- Review the available event log data, as described in [Data collection](#data-collection).
85-
- Use PowerShell, DiskPart, and sysinternals tools to inspect and manipulate the disk and volume status.
126+
```powershell
127+
Set-MPIOSetting -NewPathVerifyState Enabled
128+
Set-MPIOSetting -CustomPathRecovery Enabled
129+
```
130+
131+
- To configure intervals and timeouts for path verification, confirm appropriate settings together with your hardware vendor. Set the following values in the `HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\mpio\Parameters` subkey:
132+
133+
- `PathVerifyEnabled` (DWORD): 1 to turn on
134+
- `PathRecoveryInterval` (DWORD): Time in seconds (default: 30)
135+
- `PDORemovePeriod` (DWORD): Time in seconds (default: 20)
136+
After you change a registry value, restart the computer.
137+
138+
### Fix file system corruption or RAW disks
139+
140+
- If your disk is RAW and can't be fixed, delete and recreate the partition, then restore data from backup.
141+
- If your clustered disks have stale reservations, use `Clear-ClusterDiskReservation` at a PowerShell command prompt to clear them.
142+
143+
#### Switch DSM or remove third-party multipath drivers
144+
145+
1. If you're using third-party DSMs, use the Programs and Features control panel to uninstall them.
146+
1. Restart the computer.
147+
1. In the MPIO tool, select **Devices**, and then remove the hardware IDs. from MPIO Devices tab,
148+
1. Restart the computer again, then in the MPIO tool, re-add the hardware IDs for MSDSM.
149+
1. To make sure that Microsoft DSM claims your disks, run `mpclaim -s -d` at the Windows command prompt.
86150

87151
## Common issues and solutions
88152

89153
The following sections describe the most common issues, and how to fix them.
90154

91-
### Disks or paths missing in MPIO or OS (including after maintenance or restart)
155+
### Windows Server 2019 or 2022: VM IO performance degrades when Hyper-V uses resilient change tracking (RCT) is in use
92156

93-
#### Symptoms
157+
If you experience this issue, make sure that your computers are up to date. [October 23, 2025—KB5070884 (OS Build 20348.4297)](https://support.microsoft.com/en-us/topic/october-23-2025-kb5070884-os-build-20348-4297-out-of-band-9c001fdc-f0d2-4636-87bb-494a59da55d0) and subsequent updates contain a fix for this issue.
158+
159+
> [!NOTE]
160+
> After you install this update, VMs that have been backed up by using or a host-level backup application might not be able to start. To fix this issue, delete any .rct and .mrt files that are associated with the affected virtual hard disks. Then try again to start the VMs. If the issue persists, contact Microsoft Support.
161+
162+
### Some cluster disk resources remain in "Online Pending" state
163+
164+
The default `PendingTimeout` value is too low for cluster disks that have quota or File Server Resource Manager (FSRM) changes. As a result, the cluster disks don't come online. To increase this `PendingTimout` value, run the following cmdlets at the PowerShell command prompt:
165+
166+
```powershell
167+
Get-ClusterResource "<Resource_Name>" | Set-ClusterParameter PendingTimeout 300000
168+
```
169+
170+
> [!NOTE]
171+
>
172+
> - In these commands, \<Resource_Name> is the name of the disk resource.
173+
> - The value of the PendingTimeout property is measured in milliseconds. The value shown here is higher than the default value.
174+
175+
### Cluster disk resources are slow to come online
176+
177+
You might observe cluster disk resources taking three to five minutes to come online. This behavior also generates Event ID 1069 and Event ID 4874.
178+
179+
To fix this issue, follow these steps:
180+
181+
1. Review cluster logs for timeout or resource errors.
182+
1. In Failover Cluster Manager, increase the resource timeout from the default value (three minutes), to five minutes or more.
183+
1. Check for volume snapshots or FSRM quotas that delay bringing resources online.
184+
1. Clear any cluster dependencies or disk policies that might impede access to the disks.
185+
186+
### After maintenance or a restart, administrative tools don't show disks or paths
187+
188+
use the iSCSI Initiator tool to review the "Favorite Targets" list (In the search bar, type *iscsiocpl*, and then in the search results, select **iSCI Initiator**, and then select **Favorite Targets**.
94189

95190
- Disk missing in Disk Management or cluster
96191
- "mpclaim -s -d" missing LUNs
@@ -150,20 +245,7 @@ The following sections describe the most common issues, and how to fix them.
150245
Restart if prompted to do so.
151246
1. On clusters, increase disk resource timeouts to tolerate slow failover.
152247

153-
### Cluster disk online pending or resource failure
154-
155-
#### Symptoms
156-
157-
- Cluster disk stuck "Online Pending"
158-
- Event IDs 1069, 4874
159-
- Disk takes three to five minutes to come online
160248

161-
#### Resolution
162-
163-
1. Review cluster logs for timeout or resource errors.
164-
1. Increase resource timeout in Failover Cluster Manager (default is three minutes, try five minutes or more).
165-
1. Check for volume snapshots or FSRM quotas that delay bringing resources online.
166-
1. Clear any cluster dependencies or disk policies that are impeding access.
167249

168250
### 4. "Persistent Disk or Path Errors in Event Log"
169251

0 commit comments

Comments
 (0)