RTSS hooking into blacklisted applications?

Discussion in 'MSI AfterBurner Application Development Forum' started by gatecrasher, May 28, 2018.

  1. gatecrasher

    gatecrasher Member

    Messages:
    20
    Likes Received:
    0
    GPU:
    8GB GTX 1070
    The way I prefer to use RTSS is to only whitelist specific applications.
    I have Application Detection Level set as "None" on the global profile.

    If I search for "RTSS" in Process Explorer, I can see RTSS hooked into all sorts of processes that have not been whitelisted.
    Even applications which I have specifically blacklisted by creating a profile with "EnableHooking=0" are having RTSSHooks.dll or RTSSHooks64.dll injected into them.

    This causes many problems, including updating RTSS, as it hooks into system processes and does not unload when RTSS is closed.
    I then have to log out or even restart the PC before I can uninstall/update RTSS.

    I believe it could also be related to why RTSS is breaking the Windows 10 (1803) Game Bar process (it doesn't just block the overlay; it crashes GameBar.exe).

    This seems to be affecting significantly more applications now in 7.2.0 Beta 1.
     
  2. Unwinder

    Unwinder Moderator Staff Member

    Messages:
    15,405
    Likes Received:
    2,674
    Wrong. You're fundamentally misunderstanding the concepts of hooking and blacklisting. You cannot prevent CBT hook library from being injected by system into some specific process. It is normal too see it loaded into each process address space, regardless of selected application detection level for the target process. But you can disable library activity and API hook injection for desired process, that's what are you doing with profiles.
     
    Last edited: May 28, 2018
  3. gatecrasher

    gatecrasher Member

    Messages:
    20
    Likes Received:
    0
    GPU:
    8GB GTX 1070
    I accept that I'm probably misunderstanding this, and certain processes may require it for RTSS to work correctly.

    But, for example, RTSS is hooked into iCloudDrive.exe, iCloudPhotos.exe, and iCloudServices.exe
    I have added profiles for those processes with "EnableHooking=0" set, yet RTSSHooks.dll is still being loaded into them.

    In RTSS 7.1.0, if I searched for RTSS after loading it, I would get maybe 5 processes listed.
    In RTSS 7.2.0 Beta 1, I now have 30 processes listed when using the same settings, and it now seems to be hooked into almost everything that I run.
    Is that supposed to happen?
     
  4. Unwinder

    Unwinder Moderator Staff Member

    Messages:
    15,405
    Likes Received:
    2,674
    Once again, you're misunderstanding system functioning principles. CBT hooks are always injected by SYSTEM into ANY process performing some window or keyboard/mouse input activity. That is normal and you cannot control it with profiles. The profiles CAN prevent RTSSHooks library from inslalling API hooks and intercepting D3D/OGL/Vulkan libraries into the target process AFTER it get loaded into each process. But you cannot use the profiles to prevent hook library from being loaded into target process, never could and never will. Because it is not RTSS loading hooks library into processes and selecting which process to load it into and which to ignore.
    And no, there are no changes with 7.2.0 in this area. I have nothing more to add to it, we're wasting time on nothing, sorry.
     

  5. JonasBeckman

    JonasBeckman Ancient Guru

    Messages:
    17,486
    Likes Received:
    2,878
    GPU:
    MSI 6800 "Vanilla"
    Far as unloading is concerned just wait a few minutes and the .dll should have been unloaded from almost every process. Any remaining processes can usually be terminated manually if the .dll is still loaded after this.
    (Usually around five minutes, application frame host and shell experience host tend to be on the slower side whereas most other programs are pretty quick.)

    Just the OS handling it, considering how Microsoft is changing Game Bar and related functionality such as the full-screen optimization option that might be the cause of the crashes in Redstone 4 too but I don't have enough knowledge or a deep understanding of how this is set up to answer in any actual detail although with how frequently Microsoft sends CompatTelRunner.exe and automatically logs and sends crash report data and usage information (Especially for system processes.) chances are they'll fix it at some point.
    (These additions have certainly caused their share of compatibility issues going by all the posts about them.)


    EDIT: As for CBT that should be this.
    https://msdn.microsoft.com/en-us/library/windows/desktop/ms644959(v=vs.85).aspx#wh_cbthook
     
    Last edited: May 28, 2018
  6. gatecrasher

    gatecrasher Member

    Messages:
    20
    Likes Received:
    0
    GPU:
    8GB GTX 1070
    Thank you for the information on this.

    It doesn't seem to be related to Fullscreen Optimizations.
    The Game Bar works consistently without RTSS running.
    I can even start a game with RTSS running if the OSD is hidden.
    But the second that I toggle the RTSS OSD in a game, it breaks the Game Bar until I restart the computer.
     

Share This Page