Windows: Line-Based vs. Message Signaled-Based Interrupts. MSI tool.

... or another attempt to improve latencies

Little bit of theory:
*****
From "Windows Internals" by Mark Russinovich, David A. Solomon, Alex Ionescu
Line-Based vs. Message Signaled-Based Interrupts
Shared interrupts are often the cause of high interrupt latency and can also cause stability issues. They are typically undesirable and a side effect of the limited number of physical interrupt lines on a computer. For example, in the previous example of the 7-in-1 media card reader, a much better solution is for each device to have its own interrupt and for one driver to manage the different interrupts knowing which device they came from. However, consuming four IRQ lines for a single device quickly leads to IRQ line exhaustion. Additionally, PCI devices are each connected to only one IRQ line anyway, so the media card reader cannot use more than one IRQ in the first place.

Other problems with generating interrupts through an IRQ line is that incorrect management of the IRQ signal can lead to interrupt storms or other kinds of deadlocks on the machine, because the signal is driven “high” or “low” until the ISR acknowledges it. (Furthermore, the interrupt controller must typically receive an EOI signal as well.) If either of these does not happen due to a bug, the system can end up in an interrupt state forever, further interrupts could be masked away, or both. Finally, line-based interrupts provide poor scalability in multiprocessor environments. In many cases, the hardware has the final decision as to which processor will be interrupted out of the possible set that the Plug and Play manager selected for this interrupt, and there is little device drivers can do.

A solution to all these problems is a new interrupt mechanism first introduced in the PCI 2.2 standard called message-signaled interrupts (MSI). Although it remains an optional component of the standard that is seldom found in client machines, an increasing number of servers and workstations implement MSI support, which is fully supported by the all recent versions of Windows. In the MSI model, a device delivers a message to its driver by writing to a specific memory address. This action causes an interrupt, and Windows then calls the ISR with the message content (value) and the address where the message was delivered. A device can also deliver multiple messages (up to 32) to the memory address, delivering different payloads based on the event.

Because communication is based across a memory value, and because the content is delivered with the interrupt, the need for IRQ lines is removed (making the total system limit of MSIs equal to the number of interrupt vectors, not IRQ lines), as is the need for a driver ISR to query the device for data related to the interrupt, decreasing latency. Due to the large number of device interrupts available through this model, this effectively nullifies any benefit of sharing interrupts, decreasing latency further by directly delivering the interrupt data to the concerned ISR.

Finally, MSI-X, an extension to the MSI model, which is introduced in PCI 3.0, adds support for 32-bit messages (instead of 16-bit), a maximum of 2048 different messages (instead of just 32), and more importantly, the ability to use a different address (which can be dynamically determined) for each of the MSI payloads. Using a different address allows the MSI payload to be written to a different physical address range that belongs to a different processor, or a different set of target processors, effectively enabling nonuniform memory access (NUMA)-aware interrupt delivery by sending the interrupt to the processor that initiated the related device request. This improves latency and scalability by monitoring both load and closest NUMA node during interrupt completion.
*****

Now practice.
Checking for PCI devices working in MSI-mode.
Go to Device Manager. Click in menu "View -> Resources by type". Expand "Interrupt request (IRQ)" node of the tree. Scroll down to "(PCI) 0x... (...) device name" device nodes. Devices with positive number for IRQ (like "(PCI) 0x00000011 (17) ...") are in Line-based interrupts-mode. Devices with negative number for IRQ (like "(PCI) 0xFFFFFFFA (-6) ...") are in Message Signaled-based Interrupts-mode.

Trying to switch device to MSI-mode manually.
You must locate device`s registry key. Invoke device properties dialog. Switch to "Details" tab. Select "Device Instance Path" in "Property" combo-box. Write down "Value" (for example "PCI\VEN_1002&DEV_4397&SUBSYS_1609103C&REV_00\3&11583659&0&B0"). This is relative registry path under the key "HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Enum\".

Go to that device`s registry key ("HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Enum\PCI\VEN_1002&DEV_4397&SUBSYS_1609103C&REV_00\3&11583659&0&B0") and locate down the subkey "Device Parameters\Interrupt Management". For devices working in MSI-mode there will be subkey "Device Parameters\Interrupt Management\MessageSignaledInterruptProperties" and in that subkey there will be DWORD value "MSISupported" equals to "0x00000001". To switch device from legacy- to MSI-mode just add these subkey and value.

Before adding these key and value (or changing "MSISupported" to "0x00000001" in case subkey and value already exist) you have to perform safety steps like doing backup (creating system restore point at least).

Do tweak one device -> reboot and check (1) if it is displayed in Device Manager as correctly working device; (2) if its IRQ became negative -> if no (1) and no (2) then either remove subkey "MessageSignaledInterruptProperties" (if you added it) or change "MSISupported" to "0x00000000" and reboot.

Theoretically if device driver (and platform = chipset) unable to perform in MSI-mode it should ignore mentioned subkey and value. But who knows...

Trying to switch device to MSI-mode with MSI utility v3 (run as administrator)
http://www.mediafire.com/file/ewpy1p0rr132thk/MSI_util_v3.zip/file
MD5 hash for zip-file: 8424509737CEDBDE4BA9E9A780D5CE96

Spoiler

Change log:
- showing multiple IRQs for device (previous version showed only first of multiple IRQs);
- extending the details pane (under the grid) for the selected device to show more properties - PNP and PCI ones;
- replacing WMI with Setup API for retrieving device properties - so all info should be available on Win7/Win8 rigs;
- replacing registry names of devices with normal PNP display names from Setup API;
- adding the ability to launch the registry editor with selected device instance by double click on the device.

Edit: If you have old rig and there are no devices working in MSI-mode, then may be you should not experiment with this tweak.
As this tweak is risky then you probably should not tweak devices which can prevent Windows to boot (by their failure) - SATA controllers, videocards. Especially if they do not share their IRQ with another devices.

Edit: Troubleshooting topic with info from "Windows Internals":

Spoiler

Accepting the Boot and Last Known Good
Besides starting services, the system charges the Sevice Control Manager (SCM) with determining when the system’s registry configuration, HKLM\SYSTEM\CurrentControlSet, should be saved as the last known good control set. The CurrentControlSet key contains the Services key as a subkey, so CurrentControlSet includes the registry representation of the SCM database. It also contains the Control key, which stores many kernel-mode and user-mode subsystem configuration settings. By default, a successful boot consists of a successful startup of auto-start services and a successful user logon. A boot fails if the system halts because a device driver crashes the system during the boot or if an auto-start service with an ErrorControl value of SERVICE_ERROR_SEVERE or SERVICE_ERROR_CRITICAL reports a startup error.

The SCM obviously knows when it has completed a successful startup of the auto-start services, but Winlogon (%SystemRoot%\System32\Winlogon.exe) must notify it when there is a successful logon. Winlogon invokes the NotifyBootConfigStatus function when a user logs on, and NotifyBootConfigStatus sends a message to the SCM. Following the successful start of the auto-start services or the receipt of the message from NotifyBootConfigStatus (whichever comes last), the SCM calls the system function NtInitializeRegistry to save the current registry startup configuration.

Windows maintains several copies of CurrentControlSet, and CurrentControlSet is really a symbolic registry link that points to one of the copies. The control sets have names in the form HKLM\SYSTEM\ControlSetnnn, where nnn is a number such as 001 or 002. The HKLM\SYSTEM\Select key contains values that identify the role of each control set. For example, if CurrentControlSet points to ControlSet001, the Current value under Select has a value of 1. The LastKnownGood value under Select contains the number of the last known good control set, which is the control set last used to boot successfully. Another value that might be on your system under the Select key is Failed, which points to the last control set for which the boot was deemed unsuccessful and aborted in favor of an attempt at booting with the last known good control set. Figure 4-15 displays a system’s control sets and Select values.

NtInitializeRegistry takes the contents of the last known good control set and synchronizes it with that of the CurrentControlSet key’s tree. If this was the system’s first successful boot, the last known good won’t exist and the system will create a new control set for it. If the last known good tree exists, the system simply updates it with differences between it and CurrentControlSet. Last known good is helpful in situations in which a change to CurrentControlSet, such as the modification of a system performance-tuning value under HKLM\SYSTEM\Control or the addition of a service or device driver, causes the subsequent boot to fail. Users can press F8 early in the boot process to bring up a menu that lets them direct the boot to use the last known good control set, rolling the system’s registry configuration back to the way it was the last time the system booted successfully.

So if you choose to try and switch PCI device into the message-mode, you better do add "MSISupproted" registry value to one device and reboot. If Windows failed to boot you restart and press F8 and choose boot option "last known good configuration" (which must be the configuration without "MSISupported").

Edit: Improper Propagation of SATA Message Signaled Interrupts on AMD SB950

Spoiler

Description
The SATA Message Signaled Interrupt (MSI) from higher port numbers may not properly propagate to port 0 before being sent to the driver.

Potential Effect on System
On a platform with SATA MSI enabled and activity occurring on SATA devices connected to three or more ports, if the failing condition occurs, the driver will not receive the interrupt and the system will hang.

Suggested Workaround
Do not expose the MSI capability in the SATA controller. This change is implemented in CIMx version 1.1.0.2.

Fix Planned
No

Edit: Here is Intel`s paper (thanks to n1hilist):
http://www.intel.co.za/content/dam/...hite-papers/msg-signaled-interrupts-paper.pdf

Edit: Easy way to measure ISRs and DPCs
https://forums.guru3d.com/threads/simple-way-to-trace-dpcs-and-isrs.423884/

Edit: Dedicated thread about interrupts in Windows
https://forums.guru3d.com/threads/a-bit-detailed-info-about-interrupts-in-windows.424677/

Edit: External Interrupts in the x86 system. Part 1. Interrupt controller evolution
https://habr.com/en/post/446312/

Edit: Use this site to check hash value for files in this post - https://hash.online-convert.com/md5-generator
Even if it gives wrong hashes using the same site proves the un-compromised state.

 

So this has no effect on the IRQ's for the Numeric Data Processor and System CMOS/Real time clock? Both of these use IRQ's.

 

Are these the PCI devices in your rig? At mine they are ISA ones.

 

Great write up, although i found it somewhat difficult to follow and re-read it many times as i was confused to what the main "wanted" goal was and what was to be created.

This worked for my video card but not asus xonar essence stx audio device. I have the latest xonar software with no addons. I have always believed some of my problems (latency, DPC, etc.) has been related to his.. or maybe im just that stupid to think so i dont know. However, i'd thought i'd try. I had initially messed up by not creating the "MessageSignaledInterruptProperties" subkey first before adding "MSIsupported". I went back and re-read the post and seen the mistake which then after a reboot showed that this worked for my video card.

IS there something that prevents the change to MSIsupported mode or a software conflict/reason this wont work as like the USB 2 controllers you tried?

 

It`s my clumsy english. Sorry.

As for MSI-mode, device itself has to use (implement) MSI capability, and its driver too.

And what about network card and SATA controller in your rig? I suspect that video card is not the source of mass interrupt bursts...

 

Network adapter is Intel 82579V. It is already at a -2 IRQ. On a side note if audio wasn't the problem then either this is or maybe usb. I notice that if i mess with the performance settings or interruption/filter settings my net has extreme amount of drops. Youtube videos stop loading, my games freeze periodically as if i lose my connection and regain it. I have read so much into it and have not found the cause still after 6 months. My net is good, my modem is fine as well (just got firmware update i think yesterday even).

TBH i havent even looked at my SATA controller. Its in Line-based mode.
I will look at it tonight when i get time to mess with things again.

 

Have you tried:
- tweaking NetworkThrottlingIndex;
- tweaking TCP/IP (there is recommendation to enable TCP RSS when NIC is working in MSI-mode);
- disabling CPU core parking;
- this thread
???

 

I have slowly got into Von's page on performance tweaks. Many i've done before and or found else where so i found myself either repeating or skipping.


I monkeyed with tonight with the IRQ and upon reboot i got a blue screened before windows could log in. I ended up just reinstalling windows because i did not have system restore on that partition as i do with this one; why i dont remember.

And with a new fresh install i can hopefully nail what is causing my problems by introducing 1 thing at a time.

I did core parking and the network throttling i put to 0.

This maybe unheard off but by god i believe its keyboard or mouse related but there is no way to really test.No way mean i dont have replacements or software that i know of to find problems. I have g100 keyboard and g500 mouse.
It seems that my disconnects only happen during me actually using my hardware. Example me spectating in call of duty or bf3 or tf2i have yet to see this same phenomenon happen. My disconnects and by this i mean my game freezes and connection drops and returns are made worse by excessive input from me. So putting the peices together makes me rule out internet issues however it acts just like an internet issue. Hell it reminds of the House episode where they believe its cancer but every test shows that is not cancer. Strange.

I have no idea gentlemen. I am ready to slap this on ebay, all of it, and say the hell with it. I'll build from the ground up when i have time.

In vons thread he goes into device manager to disable stuff, so i did to. Looking under device manager i see x4 HID complaint keyboard and same for mouse. I see "virtual keyboard" and "virtual mouse". Which makes me wonder, but that about it.

I dont know. I would seriously pay someone to sit at my computer and do all that is necessary to find out why i can't use it.

 

Definitely you need network troubleshooting guru. May be from your internet provider.

 

Yea but im not sure if they would find anything. I'm at the point now i dont care anymore-i've given up.

To make my day even better my keyboard decided not to work today in its current usb slot, so i had to change usb slots t type this messsage. Why?...

 

Generic recommendation for not working USB ports is to shutdown PC, remove power cable for 5-10 minutes. Or try powered USB hub.

As for network, imo troubles are 20% of internal hardware-software and 80% of external network protocol ones. Especially when xDSL modem involved.
Btw, you can try discrete NIC, if you suspect your (most probably integrated) one is culprit.

 

I believe that i might have fixed the issue, fingers are crossed. If it isn't then i truly will just take a 12 gauge and light this whole thing up.

Two issues: 1) turned modem into a bridge via NAPT mode disabled; known issue with my modem (surfboard from motorola).
2) Keep getting usb3 power surge message. Switched 1 input to FP usb and the other to the back. So far working.

The strangest thing happened to me as well. I went through the bios to see if maybe it was power related, turned everything to stock voltages as i had been using part of an OC profile but had de-clocked my cpu long ago just never returned the voltages. Log into windows to find that my mouse is super laggy, my cpu usage is 80 percent, my ram is almost maxed at 8gigs. Logitech gaming software was using over 2 gigs of ram... all hell broke loose. I went back and re did an overlock on my system said why the hell not after i couldn't fix the damn thing lets fry it.
Well it works now.

Uhm sorry i have derailed this thread. I believe i noticed some* maybe some improvement with these tweaks. I will re-do them on my fresh install of windows.But the SATA controller is off limits.
Thanks

 

Yeah, that was USB troubleshoot # 1 - use back panel ports instead front panel ones.

 

http://i.imgur.com/khBUUqp.png
Am I doing this correctly? Because if I am, I can't even get this to work on my Audigy. Ah well, no harm done.

 

Yeah, seems correctly. Audio-card is not the source interrupts bursts, I assume. SATA controllers, USB-controllers, NICs are...

 

Hi man, I have do all you say in this guide, only 3:
Intel(R) 7500/5520/5500/X58 I/O Hub PCI Express Root Port 1 - 3408
Intel(R) 7500/5520/5500/X58 I/O Hub PCI Express Root Port 7 - 340E - 340e
Intel(R) 7500/5520/5500/X58 I/O Hub PCI Express Root Port 3 - 340A - 340a


I can't make in msi-mode, Why You know?

 

Probably because of this,

 

Only those 3 were switched into MSI-mode?
You see, for MSI-mode must participate: chipset, device and device drivers.

 

In this photo you can see the hardware is not in msi-mode


[IMG=http://img404.imageshack.us/img404/9222/catturaifv.jpg][/IMG]

 

is not in msi-mode and I cant change into msi-mode :bang: