How do I Add a PCI Device to a Virtual Machine

vSphere allows you to easily assign multiple PCI passthrough devices to a virtual machine without specifying an exact physical device on a particular ESXi host.

You can connect to the guest operating system of a virtual machine all PCI devices that are configured on an ESXi host and made available for passthrough.

You can also take advantage of the vSphere Distributed Resource Scheduler (DRS) feature. vSphere DRS can move a virtual machine to a compatible ESXi host when the virtual machine powers on.

PCI vSphere DirectPath I/O devices

vSphere DirectPath I/O devices allow you to specify the physical location of the devices that you want to add to a virtual machine. vSphere DirectPath I/O allows a virtual machine to access directly the physical PCI and PCIe devices connected to a specific host. This way you can directly access devices, such as high-performance graphics or sound cards. You can connect each virtual machine to up to sixteen PCI devices.

You configure PCI devices on an ESXi host to make them available for passthrough to a virtual machine. See the vSphere Networking documentation. However, you must not enable PCI passthrough for ESXi hosts that are configured to boot from USB devices.

When PCI vSphere DirectPath I/O devices are made available to a virtual machine, you cannot perform certain operations on the virtual machine. These operations include suspending, migration with vMotion, and taking or restoring snapshots of the virtual machine.

vSphere Enhanced DirectPath I/O devices

vSphere Enhanced DirectPath I/O devices are an improvement that allows hardware device vendors to extend the capabilities of their devices when used in virtual machines. For example, if implemented by the hardware vendor, an Enhanced DirectPath I/O device might support a virtual machine migration with vMotion or suspend and resume operations. Enhanced DirectPath I/O devices might also have configuration parameters associated with them and you can view and change these configuration parameters when you add such PCI devices to a virtual machine. The configuration parameters, if implemented by the vendor, can help you further specify and select particular hardware devices in accordance with the amount of the resources available on the device or their type.

vSphere Enhanced DirectPath I/O devices allow vSphere DRS to identify a host within the cluster that has an available device with the required resources.

PCI vSphere Dynamic DirectPath I/O devices

vSphere Dynamic DirectPath I/O devices allow you to specify the device type that you want to add to a virtual machine. vSphere Dynamic DirectPath I/O provides you with the ability to assign multiple PCI passthrough devices to a virtual machine. vSphere Dynamic DirectPath I/O allows vSphere DRS to identify a host within the cluster that has an available device with the same vendor and model name.

Vendor Device Groups

Vendor Device Groups is a collection of two or more hardware devices that you can allocate together to a virtual machine. By using Vendor Device Groups, other properties can be considered when allocating devices to a virtual machine. For example, one could ensure that a pair of devices are always allocated on the same PCI switch. Without Vendor Device Groups, each device will be allocated and assigned to a virtual machine independently of one another.

Note: You can only add and remove the entire Vendor Device Groups entity and not an individual device of that group.

If you select a Vendor Device Groups that has a NIC, you must add an appropriate network for the NIC in that Vendor Device Groups. The network that you add is one of the networks that you configure for the SR-IOV NICs to be attached to.

NVIDIA GRID GPU devices

If an ESXi host has an NVIDIA GRID GPU graphics device, you can configure a virtual machine to use the NVIDIA GRID virtual GPU (vGPU) technology.

NVIDIA GRID vGPU devices optimize complex graphics operations and make them run at high performance without overloading the CPU. By using NVIDIA GRID vGPU, you can share a single physical GPU among multiple virtual machines as separate vGPU-enabled passthrough devices.

Starting with vSphere 7.0 Update 2, you can configure a virtual machine to use the NVIDIA Multi-Instance GPU (MIG) feature. By using NVIDIA MIG, you can securely partition applicable GPUs into separate GPU instances. Each GPU instance has dedicated resources, such as memory, memory caches, and compute cores. If a GPU is in MIG mode, you can assign unique vGPU profile names to a virtual machine. VMware will create GPU and compute instances automatically, so you should not create them manually.

Note: When you add a PCI device to a virtual machine, the full memory size of the virtual machine is automatically reserved.

Prerequisites

Verify that you have the privileges that you need for the task that you plan to perform.

If you plan to add a PCI device when you edit a virtual machine, verify that you have the Virtual machine.Change Configuration.Add or remove device privilege.
If you plan to increase the memory reservation when you edit a virtual machine, verify that you have the Virtual machine.Change Configuration.Change resource privilege.
If you plan to reduce the virtual machine memory when you edit a virtual machine, verify that you have the Virtual machine.Change Configuration.Change Memory privilege.

Power off the virtual machine.
To use Dynamic DirectPath I/O, verify that the virtual machine is compatible with ESXi 7.0 or later.
To use DirectPath, verify that Intel Virtualization Technology for Directed I/O (VT-d) or AMD I/O Virtualization Technology (IOMMU) is enabled in the host BIOS.
Verify that the PCI devices are connected to the host and marked as available for passthrough. If your ESXi host is configured to boot from a USB device, or if the active coredump partition is configured to be on a USB device or SD cards connected through USB channels, deactivate the USB controller for passthrough. VMware does not support USB controller passthrough for ESXi hosts that boot from USB devices or SD cards connected through USB channels. A configuration in which the active coredump partition is configured to be on a USB device or SD card connected through USB channels is also not supported. For information, see http://kb.vmware.com/kb/1021345.
To use Enhanced DirectPath I/O, verify that:
- The virtual machine is compatible with ESXi 8.0 or later.
- You download and install special drivers provided by the hardware vendor.
To use NVIDIA GRID vGPU graphic devices:
- Verify that an NVIDIA GRID vGPU graphic device with an appropriate driver is installed on the host. See the VMware ESXi Upgrade documentation.
- Verify that the virtual machine is compatible with ESXi 6.0 and later.
To add multiple NVIDIA GRID vGPUs to a virtual machine:
- Verify that the virtual machine is compatible with ESXi 6.7 Update 2 and later.
- Use only NVIDIA vGPU profiles with a maximum frame buffer.
- Only Q-series and C-series vGPU types are supported.
To use Vendor Device Groups, verify that:
- The virtual machine is compatible with ESXi 8.0 or later.

Procedure

Add a PCI device to a virtual machine when you deploy a virtual machine or edit an existing virtual machine.

Option	Action
Create a new virtual machine	Select any inventory object that is a valid parent object of a virtual machine and click Actions. Click New Virtual Machine. On the Select a creation type page, select Create a new virtual machine, and click Next. Navigate through the pages of the wizard. On the Customize hardware page, click the Virtual Hardware tab.
Edit a virtual machine	Select any inventory object that is a valid parent object of a virtual machine and click Actions. Click Edit Settings. Click the Virtual Hardware tab.

On the Virtual Hardware tab, click the Add New Device button.
From the drop-down menu, under Other Devices, select PCI Device.
The Device Selection window that contains a list of devices appears.

Select the PCI device to connect to the virtual machine and click Select.

Option	Action
DirectPath I/O Dynamic DirectPath I/O Enhanced DirectPath I/O	Select a PCI device from the list and click Select. The PCI device appears in the Virtual Hardware device list. Expand the New PCI device section and if present, enter the configuration parameters for the PCI device. To remove the PCI device, click Remove. The device is removed from the list. To add another PCI device, click the Add Device button or undo the operation by clicking Undo.
NVIDIA GRID vGPU	Select a PCI device from the list and click Select. The PCI device appears in the Virtual Hardware device list. Expand the New PCI device section. From the vGPU Profile drop-down menu, select the NVIDIA GRID vGPU passthrough device to connect to the virtual machine. To remove the NVIDIA GRID vGPU passthrough device, click Remove. The device is removed from the list. To add another NVIDIA GRID vGPU passthrough device, click the Add Device button or undo the operation by clicking Undo. Note: You can add only one NVIDIA GRID vGPU device in MIG mode to a virtual machine.
Group	Select a Vendor Device Groups entry from the list and click Select. The Vendor Device Group devices appear in the Virtual Hardware device list. Expand the New Vendor Device Group section. If you add a Vendor Device Group that has a NIC, add an appropriate network. From the Network drop-down menu click Browse, and select an appropriate network. To remove the Vendor Device Group, click the vertical ellipsis icon and click Remove device. The Vendor Device Group is removed from the list. To undo the operation, click Undo.

Option

Action

DirectPath I/O

Dynamic DirectPath I/O

Enhanced DirectPath I/O

Select a PCI device from the list and click Select.
The PCI device appears in the Virtual Hardware device list.
Expand the New PCI device section and if present, enter the configuration parameters for the PCI device.
To remove the PCI device, click Remove.
The device is removed from the list.
To add another PCI device, click the Add Device button or undo the operation by clicking Undo.

NVIDIA GRID vGPU

Select a PCI device from the list and click Select.
The PCI device appears in the Virtual Hardware device list.
Expand the New PCI device section.
From the vGPU Profile drop-down menu, select the NVIDIA GRID vGPU passthrough device to connect to the virtual machine.
To remove the NVIDIA GRID vGPU passthrough device, click Remove.
The device is removed from the list.
To add another NVIDIA GRID vGPU passthrough device, click the Add Device button or undo the operation by clicking Undo.

Note: You can add only one NVIDIA GRID vGPU device in MIG mode to a virtual machine.

Group

Select a Vendor Device Groups entry from the list and click Select.
The Vendor Device Group devices appear in the Virtual Hardware device list.
Expand the New Vendor Device Group section.
If you add a Vendor Device Group that has a NIC, add an appropriate network.
From the Network drop-down menu click Browse, and select an appropriate network.
To remove the Vendor Device Group, click the vertical ellipsis icon and click Remove device.
The Vendor Device Group is removed from the list.
To undo the operation, click Undo.

Click Next and complete the wizard.
Power on the virtual machine.
The connected PCI devices appear:
- On the Hardware tab of the Edit Settings wizard.
- On the Summary tab in the PCI Devices panel.
You can clone the virtual machine with the PCI passthrough devices.