Just recently I have been having a problem with the Steam overlay whenever the RTSS overlay is active. What is strange is that it works fine in some Steam games, but in others I am able to get either the RTSS overlay or the Steam overlay but not both. The current issue I am having is concerning a game called ARK: Survival Evolved. Up until the last patch everything worked fine but now as I said, I am unable to use the Steam overlay when the RTSS overlay is active. I have tried Custom Direct3d Support and Stealth Mode, and neither makes a difference. Even when I disable the On-Screen Display the Steam overlay will not work. The Steam overlay will only work when I set the Application Detection to "None" for ShooterGame.exe. I have tried both RTSS 6.2.0 and 6.4.0 with the same results.
Think that you need to write a email to Battleye support to put RTSS on whitelist like I did for Arma 3.
That's the thing though, the RTSS overlay shows up just fine. The problem is that I am unable to have both the RTSS overlay and the Steam overlay at the same time, it's like the RTSS overlay is over-riding the Steam overlay. I'm not sure if that is even a BattlEye specific issue. The part that confuses me is I'm able to run both in other Steam games just fine.
It does seem to be a BattlEye issue as BattlEye is blocking RTSSHooks64.dll. May I ask who you contacted at BattlEye in order to get the file added to the whitelist?