Skip to content

Commit 3c9208b

Browse files
v-tappelgatev-tappelgate
committed
New article for AB 6819
1 parent 04a9494 commit 3c9208b

1 file changed

Lines changed: 306 additions & 0 deletions

File tree

‎support/windows-server/virtualization/troubleshoot-hyper-v-gpu-assignment-partitioning-passthrough-issues.md‎

Lines changed: 306 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,306 @@
1+
---
2+
title: Troublehooting guidance for Hyper-V GPU assignment, partitioning, and passthrough
3+
description: Troubleshoot issues that occur when configuring GPU passthrough for virtual machines by using either Direct Device Assignment (DDA) or GPU partitioning (GPU-P).
4+
ms.date: 08/11/2025
5+
manager: dcscontentpm
6+
audience: itpro
7+
ms.topic: troubleshooting
8+
ms.reviewer: kaushika, jeffhugh, v-appelgatet
9+
ms.custom:
10+
- sap:virtualization and hyper-v\high availability virtual machines
11+
- pcy:WinComm Storage High Avail
12+
---
13+
# Hyper-V GPU assignment, partitioning, and passthrough troubleshooting guidance
14+
15+
_Applies to:_   Windows Server 2025, Windows Server 2022, Windows Server 2019, Windows 11, and Windows 10
16+
17+
This article provides guidance for troubleshooting issues related to GPU passthrough in a Hyper-V environment. The article includes a checklist for troubleshooting, a description of known issues, and instructions for resolving common problems that you might encounter setting up host servers to provide GPU passthrough or configuring virtual machines (VMs) to use GPU passthrough.
18+
19+
## Troubleshooting checklist
20+
21+
The following checklist provides the general troubleshooting steps that you should follow for any GPU passthrough issue. Many of the issues you might see originate as problems with hardware compatibility, operating system compatibility, drivers, or basic configuration. If these steps don't resolve your issue, continue to the "Common issues and solutions" section.
22+
23+
### Step 1: Make sure that the functionality that you want is supported
24+
25+
- If you want to use live migration for VMs that use GPU acceleration, you have to use GPU-P and Windows Server 2025 or a newer version on the host computer. For more information about the differences in capabilities between DDA and GPU-P, see [Plan for GPU acceleration in Windows Server](/windows-server/virtualization/hyper-v/plan/plan-for-gpu-acceleration-in-windows-server).
26+
27+
- If you want to use failover clusters and deploy GPUs by using DDA, you have to use Windows Server 2025. For more information, see [Use GPUs with Discrete Device Assignment in clustered VMs](/windows-server/virtualization/hyper-v/deploy/use-gpu-with-clustered-vm?tabs=windows-admin-center&pivots=windows-server).
28+
29+
- You cannot assign the same GPU by using both DDA and GPU-P.
30+
31+
- If you want to monitor GPU usage in Task Manager, the host computer must run Windows Server 2022 or a later verion.
32+
33+
- DDA and GPU-P are not supported on desktop-class hardware or client operating systems such as Windows 10 or 11 Pro.
34+
35+
- If you want to use DDA to assign GPUs, the host computer has to run Windows Server 2016 or a newer version.
36+
37+
- If you want to use GPU-P, the host computer has to run Windows Server 2025 or a newer version.
38+
39+
- Currently, you can assign only a single GPU partition to a VM. Both the VM and the GPU partition have to be on the same host computer.
40+
41+
### Step 2: Check that server and GPU hardware support GPU passthrough
42+
43+
Make sure that the host computer uses server-class hardware and includes a supported CPU, motherboard, and GPU. If you're using GPU-P on clustered host computers, the processors must be capable of Input/Output Memory Management Unit (IOMMU) DMA bit tracking (this requirement includes processors that support Intel VT-D or AMD-Vi).
44+
45+
Review the vendor documentation for the hardware and confirm that it supports Hyper-V GPU-P or DDA, as needed. You can use vendor-provided scripts, such as `SurveyDDA.ps1` from Microsoft's open source GitHub repository [https://github.com/MicrosoftDocs/Virtualization-Documentation/tree/main/hyperv-tools/DiscreteDeviceAssignment](https://github.com/MicrosoftDocs/Virtualization-Documentation/tree/main/hyperv-tools/DiscreteDeviceAssignment).
46+
47+
> [!IMPORTANT]
48+
> At this time, the following GPUs support GPU-P:
49+
>
50+
> - NVIDIA A2
51+
> - NVIDIA A10
52+
> - NVIDIA A16
53+
> - NVIDIA A40
54+
> - NVIDIA L2
55+
> - NVIDIA L4
56+
> - NVIDIA L40
57+
> - NVIDIA L40S
58+
>
59+
> For more information, see [NVIDIA Virtual GPU Software User Guide](https://docs.nvidia.com/grid/latest/grid-vgpu-user-guide/index.html).
60+
61+
> [!NOTE]
62+
> Check the vendor information for any compatibility restrictions that apply to the GPU.
63+
64+
For more information about host computer prerequisites for DDA, see [Deploy graphics devices by using Discrete Device Assignment: Prerequisites](/windows-server/virtualization/hyper-v/deploy/deploying-graphics-devices-using-dda#prerequisites).
65+
66+
For more information about host computer prerequisites for GPU-P, see [GPU Partitioning: Requirements](/windows-server/virtualization/hyper-v/gpu-partitioning#requirements).
67+
68+
If your hardware does not support the GPU passthrough method that you want to use, take steps to upgrade the hardware. Incompatible hardware can cause a variety of issues when you try to configure or use GPUs.
69+
70+
### Step 2: Check the host computer operating system and BIOS or UEFI settings
71+
72+
1. Make sure that the Hyper-V server role is installed and enabled on the host computer, and that you have administrative permissions for the computer.
73+
74+
1. Check the BIOS/UEFI configuration of the host computer.
75+
- If you're using GPU-P, make sure that [Single Root I/O Virtualization (SR-IOV)](/windows-hardware/drivers/network/overview-of-single-root-i-o-virtualization--sr-iov-) and VT-d/IOMMU are enabled.
76+
- If you're using DDA, consider enabling SR-IOV. This setting might resolve some issues.
77+
> [!IMPORTANT]
78+
> If you change any BIOS or UEFI settings, restart the computer before you continue.
79+
80+
### Step 4: Check the versions of the guest operating systems
81+
82+
Make sure that the VMs use operating systems that support GPU passthrough.
83+
84+
- DDA is supported for Generation 1 or 2 VMs. The guests supported include:
85+
- Windows 10 or later
86+
- Windows Server 2016 or later
87+
88+
For more information about non-Windows VMs, see [Supported Linux and FreeBSD virtual machines for Hyper-V on Windows Server and Windows](../supported-linux-and-freebsd-virtual-machines-for-hyper-v-on-windows.md).
89+
90+
- GPU-P is supported for Generation 2 VMs that use the following operating systems:
91+
- Windows 10 or later
92+
- Windows 10 Enterprise multi-session or later
93+
- Windows Server 2019 or later
94+
- Linux Ubuntu 18.04 LTS, Linux Ubuntu 20.04 LTS, Linux Ubuntu 22.04 LTS
95+
96+
### Step 5: Check Windows Update status
97+
98+
Make sure that the host computer and all VMs all have relevant Windows updates.
99+
100+
If you installed any updates, restart the affected computers.
101+
102+
### Step 6: Check the drivers and GPU licensing
103+
104+
To use GPU passthrough, you have to have the correct software and the correct license. The drivers you need might differ depending on whether you're using DDA or GPU-P and on what functionality you want to use. For example, to use live migration with GPU-P, you must use the driver that's included in the NVIDIA vGPU Software v18.x or later, and you must have a license that supports GPU-P.
105+
106+
License or driver problems can cause a variety of symptoms, including the following:
107+
108+
- The Hyper-V or Windows Admin Center does not list GPUs as available for provisioning.
109+
- The Hyper-V or Windows Admin Center lists GPUs as "Ready for DDA assignment," but not for GPU-P.
110+
- Device manager on the VMs or host computer lists the GPUs as disabled or unrecognized.
111+
- You see error message that include error code 48F or error code 003.
112+
- You cannot configure a GPU pool for GPU-P.
113+
114+
To check the drivers and licensing, follow these steps:
115+
116+
1. Review the GPU vendor documentation and confirm that you have the correct and most recent GPU drivers installed on both the host computer and the VMs.
117+
118+
> [!IMPORTANT]
119+
> NVIDIA's vGPU drivers do not update or replace datacenter version drivers (non-vGPU drivers) installed on the host computer. To use GPU-P, you have to manually remove any NVIDIA datacenter version drivers before you install the vGPU software.
120+
121+
1. If you have to make any driver changes, follow these steps:
122+
123+
1. Uninstall any incorrect drivers. For example, if you want to use GPU-P together with NVidia vGPU hardware, uninstall all non-vGPU NVidia drivers.
124+
1. Install the correct drivers.
125+
1. Restart the affected computer (host computer or VM).
126+
127+
1. Make sure that you have the correct licenses for the GPUs that you're using, that you have the correct number of licenses, and that the licenses are installed or activated correctly.
128+
129+
> [!NOTE]
130+
> For more information about NVIDIA licenses, see [NVIDIA License System User Guide](https://docs.nvidia.com/license-system/latest/nvidia-license-system-user-guide/index.html).
131+
132+
1. To verify that the GPU is now available, open a Windows PowerShell command prompt window on the host computer, and then run the following command:
133+
134+
```powershell
135+
Get-VMHostPartitionableGpu
136+
```
137+
138+
> [!NOTE]
139+
> If the GPU is still not available, see [After you update Windows, the GPU doesn't function or appears as an unknown device](#after-you-update-windows-the-gpu-doesnt-function-or-appears-as-an-unknown-device).
140+
141+
### Step 7 Check your administration tools
142+
143+
- Windows Admin Center:
144+
- Make sure that you've installed the latest version of Windows Admin Center.
145+
- Make sure that you've installed the **GPUs** extension, version 2.8.0 or later.
146+
- Windows Powershell: Make sure that you've installed the Hyper-V module.
147+
148+
### Step 8 Check the VM configuration
149+
150+
If you're using GPU-P, see [Partition and assign GPUs to a virtual machine](/windows-server/virtualization/hyper-v/partition-assign-vm-gpu?tabs=windows-admin-center) for detailed information about how to configure the GPU and how to configure VMs. That article also provides information about common issues that you might encounter while configuring or assigning GPU partitions.
151+
152+
If you're using DDA to assign GPUs, follow these steps for each VM to make sure that the VMs are correctly configured.
153+
154+
1. Sign in to the Hyper-V host as an administrator.
155+
156+
1. Open an elevated PowerShell prompt, and then run the following command.
157+
158+
```powershell
159+
Get-VM -Name <VMName>
160+
```
161+
162+
> [!NOTE]
163+
> In this command, \<VMName> is the name of the VM that you're checking.
164+
165+
1. Check that the settings listed in the following table have the following values.
166+
167+
| Setting | Value |
168+
| --- | --- |
169+
| `Automatic Stop Action` | **TurnOff** |
170+
| `GuestControlledCacheTypes` | **True** |
171+
| `LowMemoryMappedIoSpace` | **3000000000** |
172+
| `HighMemoryMappedIoSpace` | **33280000000** |
173+
174+
For more information, see the following articles:
175+
176+
- [Deploy graphics devices by using Discrete Device Assignment: Configure the VM for DDA](/windows-server/virtualization/hyper-v/deploy/deploying-graphics-devices-using-dda#vm-preparation-for-graphics-devices)
177+
- [Deploy devices by using Discrete Device Assignment: MMIO space](/windows-server/virtualization/hyper-v/plan/plan-for-deploying-devices-using-discrete-device-assignment#mmio-space)
178+
179+
## Common issues and solutions
180+
181+
### Windows Server 2022 Hyper-V or VMWare: Task Manager Processes tab doesn't display GPU or vGPU usage
182+
183+
Consider a system where you run Hyper-V or VMWare VMs on a Windows Server 2022 host, and the VMs are configured to use GPUs. You might encounter an issue where Task Manager doesn't display the **GPU** column on the **Processes** tab (and it's not available on the columns menu). In such cases, the VMs have defaulted to using [Windows Display Driver Model](/windows-hardware/drivers/display/windows-vista-display-driver-model-design-guide) (WDDM) 1.x, which doesn't support GPU monitoring in Task Manager.
184+
185+
To resolve this issue, follow these steps:
186+
187+
1. If you haven't already done so, make sure that the host computer and all VMs all have relevant Windows updates.
188+
189+
If you installed any updates, restart the affected computers.
190+
191+
1. On each affected VM, follow these steps to check the display driver version.
192+
1. Open a Command Prompt window and then run the `dxdiag` command.
193+
1. In the DirectX Diagnostic Tool, select the **Display** tab, and then look for **Driver Model**. This value should be WDDM 2.x or a later version.
194+
1. If the driver model is 2.x or higher, consider upgrading the host computer to Windows Server 2025. Otherwise, continue following this procedure to resolve this issue.
195+
1. On each affected VM, follow these steps to update the drivers and rebuild the performance counters:
196+
1. Uninstall the incorrect drivers.
197+
1. Install the correct drivers.
198+
1. Restart the affected VM.
199+
1. After the VM has restarted, rebuild the performance counters. At the command prompt, run the following commands, in sequence:
200+
201+
```console
202+
lodctr /R
203+
diskperf -y
204+
```
205+
206+
1. If the preceding steps didn't resolve the issue, consider upgrading the host computer to Windows Server 2025
207+
208+
### After you update Windows, the GPU doesn't function or appears as an unknown device
209+
210+
In this case, a VM suddenly can't use a GPU. Device Manager might list the GPU as disabled or as an unknown device. You might also see error message that include error code 48F or error code 003.
211+
212+
This situation typically means that a driver is corrupted, or a stale version of the driver was installed during the update. Simply reinstalling the correct driver might not resolve the problem, because stale or incorrect information might remain in the registry or system files.
213+
214+
> [!IMPORTANT]
215+
>
216+
> - First, make sure that you have already completed the steps described in the [Troubleshooting Checklist](#troubleshooting-checklist) section. If the issue persists, then use the following procedure.
217+
> - Before you use this procedure, check with your vendor to ensure that cleaning up registry information or system files is advised in your situation. If it is not, skip those steps.
218+
> - You must have administrative permissions to use this procedure.
219+
220+
To clean up this information and resolve this issue, follow these steps on the affected VM:
221+
222+
1. Restart the VM into [Safe mode](https://support.microsoft.com//windows/windows-startup-settings-1af6ec8c-4d4a-4b23-adb7-e76eef0b847f).
223+
224+
1. To enumerate the current drivers, open an elevated Command Prompt window and then run the following command:
225+
226+
```console
227+
pnputil /enum-drivers
228+
```
229+
230+
1. To remove the old drivers and clean up related system files and registry entries, follow these steps:
231+
232+
1. Uninstall any incorrect drivers. For example, if you want to use GPU-P together with NVidia vGPU hardware, uninstall all non-vGPU NVidia drivers. To do this, run the following command at the command prompt:
233+
234+
```console
235+
pnputil /delete-driver <Published Name> /uninstall /force
236+
```
237+
238+
> [!NOTE]
239+
> In this command, \<Published Name> is the name of the driver.
240+
241+
1. Use the [DevNodeClean](https://www.microsoft.com/download/details.aspx?id=42286&msockid=2984b5ddab276c5d2200a0dbaa9d6d1d) utility to remove stale device nodes.
242+
243+
1. To repair the system files, run the following commands, in sequence, at the command prompt:
244+
245+
```console
246+
sfc /scannow
247+
DISM /Online /Cleanup-Image /RestoreHealth
248+
```
249+
250+
1. Install the correct GPU drivers from the vendor.
251+
252+
1. Restart the VM.
253+
254+
1. To verify that the GPU is now available, open a Windows PowerShell command prompt window on the host computer, and then run the following command:
255+
256+
```powershell
257+
Get-VMHostPartitionableGpu
258+
```
259+
260+
### Cannot assign a GPU to a VM because of insufficient resources
261+
262+
When you are assigning a GPU to a VM, you might see errors such as the following messages:
263+
264+
> This Device Cannot Find Enough Free Resources to Use (Code:12)
265+
266+
> GPU Partition: Error Insufficient system resources exist to complete the requested service.
267+
268+
Such messages indicate that the VM does not have enough allocated Memory-Mapped IO (MMIO) space, or is otherwise misconfigured. For information about how to calculate and configure the resources that VMs need, see the following articles:
269+
270+
- For DDA, see [Deploy devices by using Discrete Device Assignment: MMIO space](/windows-server/virtualization/hyper-v/plan/plan-for-deploying-devices-using-discrete-device-assignment#mmio-space).
271+
- For GPU-P, see [Partition and assign GPUs to a virtual machine](/windows-server/virtualization/hyper-v/partition-assign-vm-gpu?tabs=windows-admin-center) and your NVIDIA vGPU documentation.
272+
273+
## Data collection
274+
275+
If you still can't resolve your issue, contact Microsoft Support or your vGPU vendor for more assistance. Keep the information that you gathered while troubleshooting and the results of your troubleshooting available to share with support personnel. In addition, gather the following information:
276+
277+
- To gather information about GPUs, run the following PowerShell command:
278+
279+
```powershell
280+
Get-PnpDevice -FriendlyName "<device-friendly-name>"
281+
```
282+
283+
- If you're using GPU-P, run the following PowerShell command on the host computer to get the details of the partitionable GPUs:
284+
285+
```powershell
286+
Get-VMHostPartitionableGpu | FL Name, ValidPartitionCounts
287+
```
288+
289+
- If you haven't already done so, note the **Driver Model** information on the **Display** tab of the DirectX Diagnostic Tool.
290+
- In Device Manager, note the properties of the affected GPU from the **General**, **Driver,** and **Location Path** tabs.
291+
- In Windows Admin Center, note the GPU status.
292+
- Not any applicable load information in Task Manager and `nvidia-smi`.
293+
294+
## More information
295+
296+
- [Plan for GPU acceleration in Windows Server](/windows-server/virtualization/hyper-v/plan/plan-for-gpu-acceleration-in-windows-server)
297+
- [GPU Partitioning](/windows-server/virtualization/hyper-v/gpu-partitioning)
298+
- [Partition and assign GPUs to a virtual machine](/windows-server/virtualization/hyper-v/partition-assign-vm-gpu)
299+
- [Deploy devices by using Discrete Device Assignment](/windows-server/virtualization/hyper-v/plan/plan-for-deploying-devices-using-discrete-device-assignment)
300+
- [Deploy graphics devices by using Discrete Device Assignment: Prerequisites](/windows-server/virtualization/hyper-v/deploy/deploying-graphics-devices-using-dda#prerequisites)
301+
- [Use GPUs with Discrete Device Assignment in clustered VMs](/windows-server/virtualization/hyper-v/deploy/use-gpu-with-clustered-vm?tabs=windows-admin-center&pivots=windows-server)
302+
303+
For more information about the NVIDIA hardware, software, and licenses, see the following articles:
304+
305+
- [NVIDIA Virtual GPU Software User Guide](https://docs.nvidia.com/grid/latest/grid-vgpu-user-guide/index.html)
306+
- [NVIDIA License System User Guide](https://docs.nvidia.com/license-system/latest/nvidia-license-system-user-guide/index.html)

0 commit comments

Comments
 (0)