Improve DPC Latency Spikes For Ada Lovelace-based GPUs in R536.67 Driver

Improve DPC Latency Spikes For Ada Lovelace-based GPUs in R536.67 Driver​

NVIDIA driver R536.67+ has improved the DPC Latency Spikes issues. But some hardware configurations may not be effectively improved.

Spoiler: Potential workaround

1. Enable Message Signaled Interrupts (MSI / MSI-X) for the Nvidia HDMI and GPU instances (From Reddit.NVIDIA). Adjust related PCI Express Clock Gating features in the motherboard BIOS (From this thread, @Astyanax)
2. Disable Hardware Accelerated GPU Scheduling (HAGS). Note: DLSS3 Frame Generation feature require HAGS.
3. Setting Nvidia's "Power Management Mode" to "Prefer Maximum Performance" in the Nvidia Control Panel. or setting Nvidia's "Power Management Mode" to "Prefer Maximum Performance" for each application using NVIDIA Profile Inspector.
4. via NVIDIA Power Management (NVPMM) tool to increase Min GDDR6X Clock.

Use Windows Performance Analyzer to Analyze DPC/ISR

About Windows Performance Toolkit: Windows Performance Toolkit Information.

1. Download Windows Performance Analyzer Tool from HERE

The first run will ask if you want to install Windows Performance Analyzer.


2. Set the run duration time and click "Start".


3. Finish.


4. After finishing, the *.elt file will be opened automatically. *.elt file path: "C:\wpaResult.etl"


5. By default displays "ms". You can set display to "us".


Demo: How to Analyze DPC/ISR with Windows Performance Analyzer


PC specifications
CPU: AMD Ryzen 9 7900
Cooler: Noctua NH-D12L
MB: ASRock B650M PG Riptide
RAM: Kingston DDR5-6000Mhz CL36 EXPO 64GB (2x32GB)
VGA: ASUS ROG Strix GeForce RTX 4090 OC 24GB
PSU: FSP HYDRO PTM PRO 1200W ATX3.0(PCIe 5.0)
Case: Phanteks Enthoo Pro 2 Closed Panel
OS: Windows 11 22H2 22621.2066
Driver: NVIDIA GeForce Game Ready R536.67
MB BIOS: 1.18 AGESA ComboAM5 1.0.0.5c

Increase Min GDDR6X Clock via NVIDIA Power Management tool
Gfrad v11.0|NVIDIA Power Management v2.25.20 - Define Power Limit for Each Game | guru3D Forums

Default

REPORTED DPCs

DPC routines are part of the interrupt servicing dispatch mechanism and disable the possibility for a process to
utilize the CPU while it is interrupted until the DPC has finished execution.
Highest DPC routine execution time (µs): 1032.112646
Driver with highest DPC routine execution time: nvlddmkm.sys - NVIDIA Windows Kernel Mode Driver, Version 536.67
, NVIDIA Corporation
Highest reported total DPC routine time (%): 0.003495
Driver with highest DPC total execution time: ntoskrnl.exe - NT Kernel & System, Microsoft Corporation
Total time spent in DPCs (%) 0.010635
DPC count (execution time <250 µs): 535025
DPC count (execution time 250-500 µs): 0
DPC count (execution time 500-10000 µs): 1324
DPC count (execution time 1000-2000 µs): 10
DPC count (execution time 2000-4000 µs): 0
DPC count (execution time >=4000 µs): 0

Increase Min GDDR6X Clock: 810 MHz

REPORTED DPCs

DPC routines are part of the interrupt servicing dispatch mechanism and disable the possibility for a process to
utilize the CPU while it is interrupted until the DPC has finished execution.
Highest DPC routine execution time (µs): 604.132954
Driver with highest DPC routine execution time: nvlddmkm.sys - NVIDIA Windows Kernel Mode Driver, Version 536.67
, NVIDIA Corporation
Highest reported total DPC routine time (%): 0.003408
Driver with highest DPC total execution time: ntoskrnl.exe - NT Kernel & System, Microsoft Corporation
Total time spent in DPCs (%) 0.008898
DPC count (execution time <250 µs): 398405
DPC count (execution time 250-500 µs): 0
DPC count (execution time 500-10000 µs): 702
DPC count (execution time 1000-2000 µs): 0
DPC count (execution time 2000-4000 µs): 0
DPC count (execution time >=4000 µs): 0

 

I updated the DPC Latency Spikes test results. Install the R536.67 driver.

 

That's hell of a thing... thanks btw.

 

10min test with 6 browsers and 5001MHz ^^ Driver 536.67 installed with NVCleaninstall, Gigabyte 4080, Windows 11 debloated, CPU c states off in bios, ASPM off

Did the same thing with 810MHz and it was shooting up to 3000us after 2min with a ton of pagefaults

 

It works best if you increase the GDDR6X frequency to half (5001 MHz). But the idle power will increase a little.

 

Yeah, a minimum of 5GHz VRAM clock seems to help keeping DPC latency below 1000 us but it also pins the GPU clock to 2GHz.

A bit? My card went from drawing ~30W to ~100W.