We need to talk about UE4 Shader compilation issues

Discussion in 'Videocards - NVIDIA GeForce Drivers Section' started by snight01, Mar 12, 2022.

  1. lukas_1987_dion

    lukas_1987_dion Master Guru

    Messages:
    636
    Likes Received:
    125
    GPU:
    RTX 3090 Gaming OC
    Switching to consoles just because UE4 games stutter? To have that medium details and no control over most options? Never! :p (UE4 stutter sucks but I can live with that, its not that bad on my end really, only entering new locations is annoying, but it goes away in 2-3 sec so..)
     
  2. Memorian

    Memorian Ancient Guru

    Messages:
    3,503
    Likes Received:
    479
    GPU:
    RTX 3080 Ti
    We'll see what happens in Callisto Protocol, i think it's the first AAA UE5 game.
     
  3. Smough

    Smough Master Guru

    Messages:
    861
    Likes Received:
    214
    GPU:
    GTX 1660
    Yeah, some can tolerate it a bit more, still, imo it's just really bad and shouldn't be "accepted".
     
  4. dr_rus

    dr_rus Ancient Guru

    Messages:
    3,355
    Likes Received:
    614
    GPU:
    RTX 3080
    I dunno if you can call it AAA but what do you expect to happen? UE5's handling of the issue is essentially the same as in UE4 at the moment.
     

  5. BmB23

    BmB23 Active Member

    Messages:
    58
    Likes Received:
    15
    GPU:
    GTX 1660 6GB
    It's not, it didn't start being a regular issue until this generation. Maybe it's always been an issue on linux due to poor porting/some inherent issue in linux graphics subsystems, but it has never been an issue on windows in the past.
     
  6. Jobert

    Jobert Active Member

    Messages:
    76
    Likes Received:
    26
    GPU:
    RTX 3080 ti
    Ok if the shader compilation in DX12 is the main thing being blamed for stuttering and hitching in Unreal then what the hell is the stuttering in DX11 or even other unreal engines from? probably 75% of unreal engine 3 and 4 games have stutter and hitching most of which occurs in the same exact spots and its easily reproducible if some oblivious twit wants to argue about it. FFS go back and play Batman Arkham Asylum and it hitches and stutters today on modern hardware just like it did when it came out way back then.
     
    Smough likes this.
  7. xV_Slayer

    xV_Slayer Active Member

    Messages:
    76
    Likes Received:
    30
    GPU:
    RTX 3090 FE
    You are clueless it seems. This has plagued UE3/4 also. Go play Borderlands 1 or 2 and tell me they don't stutter.
     
    Smough and Jobert like this.
  8. xV_Slayer

    xV_Slayer Active Member

    Messages:
    76
    Likes Received:
    30
    GPU:
    RTX 3090 FE
    You say this but then say that UE5 will fix the issue in previous posts. Which one is it?
     
  9. dr_rus

    dr_rus Ancient Guru

    Messages:
    3,355
    Likes Received:
    614
    GPU:
    RTX 3080
    I'm saying that Epic is well aware of the issue and will likely fix it in UE5 eventually.
     
  10. pSXAuthor

    pSXAuthor Member

    Messages:
    10
    Likes Received:
    1
    GPU:
    VIC2
    There are basically two steps to generating code to run on the GPU:

    1. Actual shader compilation (compiling the source code into the intermediate object code. DXBC in D3D11 (and earlier), DXIL for D3D12 and SPIR-V for Vulkan).
    2. Converting the intermediate object code into actual code for the target GPU

    1 should never happen at runtime (it can, but its never necessary - games ship the intermediate object code. In UE this is done as part of the build process and you should never have actual shader compilation happening at runtime (although it is possible if you screw up).

    2 is actually not as simple as it sounds because in addition to the shader various other state must be taken into account (for example the vertex input format, the render target format, and some other bits and pieces - this generally cannot be known until runtime). In D3D12 and Vulkan this process is fully encapsulated into what are called "pipelines" - you give the API a full description of the pipeline (shader code + relevant state) and the driver then converts the intermediate shader code into code that can actually run on the GPU. This is what the driver shader cache is for. This was all true before D3D12 too, except that there was no formal process for handing the driver all of the information at the same time - it could only be done at the moment you ask the GPU to draw something. Because of this its likely that drivers do a lot more optimization for D3D12 and Vulkan shaders than they did for D3D11 and earlier (this was the main reason for switching to this less flexible model after all), because it HAD to be fast and rough (however there definitely were hitches caused by this). In D3D12 and Vulkan you basically trade some flexibility at runtime for performance (you can still do everything you could before, its just that you might need to build many different combinations of pipelines to express everything you want to change at runtime - ideally only doing all of the expensive work in the driver once).

    In D3D12 and Vulkan best practice is to frontload this work by giving the driver all of the pipelines that you will ever use at startup. This gives plenty of time for the driver to perform this work in the background (or potentially load it from a cache).

    Last time I worked with the UE4 code it did not have any way to do this work up front, possibly it still doesn't to this day (too lazy to check :). That's probably why this issue is worse for UE4 than for other games. But this is not the fault of D3D12 or Vulkan - its just that you must architect things differently to optimally utilise these APIs, and this is quite a lot of work to retrofit to an existing engine (as you can see from some games that were fully designed for D3D12 or Vulkan that do not have these problems).

    However: you cannot point at pipeline generation for every hitch that you see in a game - you can only know by measuring. It is possible (and best practice) to completely solve this issue though.

    On consoles most of the time both 1 and 2 are done during build time - but this is simpler because the hardware is fixed.

    (and btw: I would not really call 2 "shader compiling" - its more like just the back end of a compiler + a bit of additional code generation for things like fetching vertices or converting output formats. Although I understand that for less technical people this might just be splitting hairs :).

    TLDR: same problem existed in D3D11 - but modern APIs allow more optimisation, so the driver tries harder (and the hitches can be worse if you don't do it up front, like you are supposed to).

    [edit] BTW: this might also help you to understand why disabling the driver shader cache does not have an effect on some games - they are probably doing it right - so caching will only have an effect on startup time. Any hitches you see are probably some other problem in the engine.
     
    Last edited: Jul 24, 2022

  11. jesped

    jesped Member Guru

    Messages:
    119
    Likes Received:
    19
    GPU:
    Gigabyte RTX 3070
    [​IMG]

    But maybe @PapaJohn can tell them they're doing something wrong, because in his magic machine it's buttery silky smooth.

     
    Last edited: Jul 25, 2022
    Terepin and Smough like this.
  12. aufkrawall2

    aufkrawall2 Ancient Guru

    Messages:
    2,084
    Likes Received:
    625
    GPU:
    3060 TUF
    I don't think it's mostly shader compile stutter in this game, but asset streaming and mediocre frame pacing in general without fps limiter. It's not as terrible as in Better Together or Kena, but it's still far from being great.

    Srsly, how can you think something like this isn't perceived as distinct stutter?
    [​IMG] [​IMG]

    Though the game is of course still enjoyable. Still, would be better without stutter...
     
    Smough likes this.
  13. dr_rus

    dr_rus Ancient Guru

    Messages:
    3,355
    Likes Received:
    614
    GPU:
    RTX 3080
    It's pretty clear that it's not just shader compilation which is at fault because there are stuttering issues on PS4 and PS5 as well.
    The question is though if it's even related to shader compilation at all as the game is DX11 which usually means that stuttering is likely because of assets management and not shader compilation.
    Anyone did a test running through the same areas more than once with driver shader cache enabled?
     
  14. PapaJohn

    PapaJohn Master Guru

    Messages:
    330
    Likes Received:
    87
    GPU:
    MSI Gam'Z 3080 12G
    Arkham Asylum does still hitch on my system too unless I use the ini tweaks that I've used since way back, when the game came out. Then it's smooth as silk.
     
  15. Shadowdane

    Shadowdane Maha Guru

    Messages:
    1,448
    Likes Received:
    81
    GPU:
    MSI RTX 3080 Suprim
    Is there any guide for tweaking these settings, or can they be changed?? I started Stray and the stuttering is killing me if I can minimize or eliminate it would be amazing!
     
    Last edited: Jul 25, 2022

  16. Horus-Anhur

    Horus-Anhur Ancient Guru

    Messages:
    4,677
    Likes Received:
    5,499
    GPU:
    RTX 2070 Super
    We can minimize somewhat. It varies on a game per game basis.
    This is my config, that I use on UE4 games, with moderate success.
    It tweaks streaming, memory pools, but also TAA, motion blur, dof, etc.
    You can tweak them to your preference.
    It's tweaked for nvidia cards. For AMD, a couple of settings might have to be changed, mostly the culling system.

     
  17. Cave Waverider

    Cave Waverider Maha Guru

    Messages:
    1,406
    Likes Received:
    265
    GPU:
    RTX 3090 ROG Strix
    In Stray for me, most of the hitches seem to be gone when replaying the same level. I do have the driver shader cache set to "Unlimited".
     
    dr_rus likes this.
  18. Shadowdane

    Shadowdane Maha Guru

    Messages:
    1,448
    Likes Received:
    81
    GPU:
    MSI RTX 3080 Suprim
    That goes in the Engine.ini file? Sorry haven't really messed with tweaking UE4 games before.
     
  19. Horus-Anhur

    Horus-Anhur Ancient Guru

    Messages:
    4,677
    Likes Received:
    5,499
    GPU:
    RTX 2070 Super
    Yes.
     
  20. Smough

    Smough Master Guru

    Messages:
    861
    Likes Received:
    214
    GPU:
    GTX 1660
    Yes, aside from the shader compilation and asset stuttering seen in UE4, it also packs another nice feature: Horrible frametimes that even with RivaTuner there will be a lot of variances anyway. I've seen this with all UE4 games, only Borderless helps a bit, but you may lose performance by doing that. This engine is just a nightmare for any PC gamer...
     

Share This Page