We need to talk about UE4 Shader compilation issues

They do, but some settings are incorrect.

I'm currently using this engine.ini
for people using NVidia, remember to change r.HZBOcclusio to 0

Spoiler

[/script/engine.renderersettings]
r.MipMapLODBias=0
r.SSR.Temporal=1
r.TemporalAA.Algorithm=1
r.TemporalAA.R11G11B10History=1
r.TemporalAAFilterSize=0.7
r.TemporalAAPauseCorrect=1
r.TemporalAACatmullRom=1
r.TemporalAACurrentFrameWeight=0.3
r.TemporalAASamples=4
r.TemporalAASharpness=0.7
r.PostProcessAAQuality=6
r.ScreenPercentage=100
r.TemporalAA.Upsampling=1
r.Upscale.Quality=5
r.SceneColorFringe.Max=0
r.SceneColorFringeQuality=0
r.MotionBlur.Max=0
r.LensFlareQuality=0
r.MotionBlurQuality=0
r.Vignette=0
r.Tonemapper.Quality=1
r.Tonemapper.GrainQuantization=0
r.TonemapperGamma=0
r.TessellationAdaptivePixelsPerTriangle=64
r.DefaultFeature.LensFlare=0
r.TriangleOrderOptimization=1
r.Shaders.Optimize=1
r.OptimizeForUAVPerformance=1
r.AOAsyncBuildQueue=1
r.AOComputeShaderNormalCalculation=1
r.MaxAnisotropy=16
r.VirtualTexture=1
r.FastVRam.BokehDOF=1
r.FastVRam.CircleDOF=1
r.FastVRam.CombineLUTs=1
r.FastVRam.Distortion=1
r.FastVRam.HZB=1
r.FastVRam.Histogram=1
r.FastVRam.SSR=1
r.FastVRam.ShadowPointLight=1
r.FastVRam.Tonemap=1
r.FastVRam.Upscale=1
r.DX11NVAfterMathEnabled=0
fx.ForceCompileOnLoad=1
D3D12.PSO.DiskCache=1
D3D12.PSO.DriverOptimizedDiskCache=1
D3D11.AFRSyncTemporalResources=1
D3D11.AFRUseFramePacing=1
D3D11.AdjustTexturePoolSizeBasedOnBudget=1
D3D11.AsyncDeferredDeletion=1
D3D11.ForceThirtyHz=0
D3D11.InsertOuterOcclusionQuery=1
D3D11.ResidencyManagement=1
D3D11.StablePowerState=0
D3D11.SyncWithDWM=1
D3D11.TexturePoolOnlyAccountStreamableTexture=1
D3D11.UseUpdateTexture3DComputeShader=1
D3D11.ZeroBufferSizeInMB=32
D3D12.AFRSyncTemporalResources=1
D3D12.AFRUseFramePacing=1
D3D12.AsyncDeferredDeletion=1
D3D12.ForceThirtyHz=0
D3D12.InsertOuterOcclusionQuery=1
D3D12.ResidencyManagement=1
D3D12.StablePowerState=0
D3D12.SyncWithDWM=1
D3D12.TexturePoolOnlyAccountStreamableTexture=1
D3D12.UseUpdateTexture3DComputeShader=1
D3D12.ZeroBufferSizeInMB=128
D3D12.MaximumFrameLatency=0
r.CompileShadersForDevelopment=0
r.D3D.ForceShaderConductorDXCRewrite=1
r.ShaderPipelineCache.PreOptimizeEnabled=1
r.ShaderPipelineCache.SaveUserCache=1
r.ShaderPipelines=1
r.Shaders.ZeroInitialise=1
r.Shadow.CachePreshadow=1
r.Shadow.CacheWPOPrimitives=1
r.SceneRenderTargetResizeMethod=2
r.SceneRenderTargetResizeMethodForceOverride=1
r.ShaderComplexity.CacheShaders=1
r.ShaderPipelineCache.StartupMode=3
r.ShaderPipelineCache.LogPSO=1
r.ShaderPipelineCache.SaveBoundPSOLog=1
r.ShaderPipelineCache.ReportPSO=1
r.ShaderPipelineCache.GameFileMaskEnabled=1
r.ShaderPipelineCache.LazyLoadShadersWhenPSOCacheIsPresent=1
r.ShaderPipelineCache.BatchSize=50
r.ShaderPipelineCache.PrecompileBatchSize=50
r.Vulkan.PipelineCacheFromShaderPipelineCache=
r.Shaders.Optimize=1
r.Shaders.FastMath=1
r.UseShaderCaching=1
r.UseShaderPredraw=1
FX.AllowGPUParticles=1
FX.AllowAsyncTick=1
FX.EarlyScheduleAsync=1
r.ForceAllCoresForShaderCompiling=1
r.TargetPrecompileFrameTime=-1
r.UseShaderDrawLog=1
r.UseShaderBinaryCache=1
r.PredrawBatchTime=-1
r.InitialShaderLoadTime=-1
r.AccelPredrawBatchTime=0
r.AccelTargetPrecompileFrameTime=0
FX.BatchAsync=1
FX.BatchAsyncBatchSize=1
r.Emitter.FastPoolEnable=1
FX.AllowGPUSorting=1
FX.AllowCulling=0
r.D3D.ForceDXC=1
r.GTSyncType=1
niagara.CreateShadersOnLoad=1
r.ShaderPipelineCache.AlwaysGenerateOSCache=0
r.XGEShaderCompile=1
r.XGEShaderCompile.Mode=1
r.XGEShaderCompile.Xml.BatchGroupSize=256
r.XGEShaderCompile.Xml.BatchSize=16
r.XGEShaderCompile.Xml.JobTimeout=0.500000
r.RHICmdAsyncRHIThreadDispatch=1
r.RHICmdBalanceParallelLists=2
r.RHICmdBalanceTranslatesAfterTasks=1
r.RHICmdBasePassDeferredContexts=1
r.RHICmdCollectRHIThreadStatsFromHighLevel=0
r.RHICmdDeferSkeletalLockAndFillToRHIThread=1
r.RHICmdPrePassDeferredContexts=1
r.RHICmdShadowDeferredContexts=1
r.RHICmdSpewParallelListBalance=0
r.RHICmdTranslucencyPassDeferredContexts=1
r.RHICmdUseDeferredContexts=1
r.RHICmdUseParallelAlgorithms=1
r.RHICmdVelocityPassDeferredContexts=1
r.RHICmdUseThread=1
r.RHICmdWidth=12
r.ParallelRendering=1
r.ParallelShadows=1
r.ParallelTranslucency=1
r.ParallelVelocity=1
r.SkeletalMeshLODBias=-2
r.HighQualityLightMaps=1
r.CreateShadersOnLoad=1
r.HZBOcclusion=2
r.AllowOcclusionQueries=1
r.AMDDisableAsyncTextureCreation=0
net.AllowAsyncLoading=1

[/script/engine.engine]
FrameRateLimit=144.000000
bEnableMouseSmoothing=False
r.OneFrameThreadLag=0
bAllowMultiThreadedShaderCompile=true

[ConsoleVariables]
AllowAsyncRenderThreadUpdates=1
AllowAsyncRenderThreadUpdatesDuringGamethreadUpdates=1
AllowAsyncRenderThreadUpdatesEditor=1
s.AsyncLoadingThreadEnabled=True

[TextureStreaming]
r.MeshStreaming=1
r.Streaming.Boost=2
r.Streaming.FullyLoadUsedTextures=1
r.Streaming.HLODStrategy=0
r.bForceCPUAccessToGPUSkinVerts=True
r.Streaming.FramesForFullUpdate=1
r.Streaming.UseBackgroundThreadPool=1
r.Cache.UpdateEveryFrame=1
r.Streaming.DropMips=0
r.Streaming.LimitPoolSizeToVRAM=0
r.Streaming.MaxTempMemoryAllowed=1024
r.Streaming.MinMipForSplitRequest=0
r.Streaming.HiddenPrimitiveScale=1
r.DetailMode=3
r.LODFadeTime=0.2
r.Streaming.MipBias=0
r.Streaming.UseAllMips=1
r.Streaming.UseMaterialData=1
r.Streaming.UseNewMetrics=1
r.Streaming.UsePerTextureBias=1
GeometryCache.InterpolateFrames=1
GeometryCache.OffloadUpdate=1
r.UseAsyncShaderPrecompilation=1
r.AmbientOcclusion.AsyncComputeBudget=1
r.HZB.BuildUseCompute=1
r.MultithreadedLightmapEncode=1
r.MultithreadedShadowmapEncode=1
r.RDG.AsyncCompute=1
RHI.MaximumFrameLatency=3
r.Streaming.DefragDynamicBounds=1
s.AsyncLoadingThreadEnabled=True
r.Streaming.PoolSize=2000
r.VirtualTextures=1
r.Streaming.AmortizeCPUToGPUCopy=1
r.Streaming.MaxNumTexturesToStreamPerFrame=3
r.Streaming.NumStaticComponentsProcessedPerFrame=3
s.AsyncLoadingThreadEnabled=1
s.AsyncLoadingTimeLimit=2.0
s.LevelStreamingActorsUpdateTimeLimit=1.00
s.UnregisterComponentsTimeLimit=0.00
s.AsyncLoadingUseFullTimeLimit=0
s.ContinuouslyIncrementalGCWhileLevelsPendingPurge=1
s.PriorityAsyncLoadingExtraTime=10.00
s.IoDispatcherCacheSizeMB=2048
s.LevelStreamingComponentsRegistrationGranularity=1
s.LevelStreamingComponentsUnregistrationGranularity=1
s.MaxIncomingRequestsToStall=100
s.MaxReadyRequestsToStallMB=0
s.MinBulkDataSizeForAsyncLoading=1000

 

Yea i found these lines from a PUBG subreddit, didnt put much thought into the individual lines

 

It is and most of the time doesn't do anything, I have noticed most UE4 games ignore these commands, it has to be done on dev side so it fully works.

 

Prove it, post videos with no frametime spikes stutter. Everyone would do it, even Digital Foundry would give these commands to everybody if this is the silver bullet to UE4 shader compilation stutter. I have tried all of these damn papyrus line of commands and all UE4 games behave exactly the same with or without it, the only way to solve it a lot, if not fully is with dxvk-async mod and not even that can fix some games.

 

Look i mean you can just try it yourself, u will notice your cpu usage will go to 100% during some loading screens so its def loading the shaders. Again It only works on dx11 games and wont solve the asset streaming stutter in games like jedi fallen order

Ill get a video soon

 

If you test this, it would entail monitoring how the game builds the shaders on run 1 with defaults (you’d have to wipe shader cache between runs) then run it again deleting the shader cache between with those new commands. At which point it would be a matter of checking differing DX11 UE4 games with the same approach to see if it’s a broad fix. In my experience, pretty often changing UE commands has no effect but it “seems to” help since so many of the shaders were compiled the first go around before enabling said commands — actually deleting them physically between the runs made me realize there was actually no difference. DX11 UE4 games do often handle shader compilation better than DX12 ones too in my experience though perhaps it depends. I imagine this is because that task is getting offloaded to a driver default approach of some kind, but I’m not sure.

 

Ok, i question a lot of stuff, but when I am wrong, i admit it. This works, i tried Mortal Shell and the game didn't have any stutter, smooth, super smooth, but it is so badly optimized, was choking my GPU even on Medium (I was getting 60 fps anyway, but I don't like to keep my GPU punished like that), but as for "stutter", there was none, everything was running quite well. Gonna try some other games I guess. At least this gave me better results than the other one which is a huge text wall and didn't do nothing for me.

 

... Interesting. I bookmarked these for later reference. Thanks peeps.

 

Can you use these commands together with DXVK Async? Is it worth to use these simultaneously?

 

Tried it on Sifu and didn't work, that game is a disaster, has small freezes and lags really hard, looks like it only works on games that don't have asset loading stutter.

 

I tried the unreal script in evil west (dx11 game) and it does nothing. There are shader compilation stutters as usual.

 

Could be, i think it's worth a shot, if it makes the game worse, you can always just delete them.

 

I honestly wanted to stay away from this thread, but I'll give you a few notes...
- These ShaderCache commands will only work on games that came with (correctly) built-up shader log.
- This old system based on FShaderCache class is used with shaders only and doesn't build PSOs - which is the main reason why it was replaced.
- The whole FShaderCache has been deprecated in UE4.25(+/- I'm too lazy to check the exact version) and replaced with FShaderPipelineCache, which doesn't even use any of these commands.

 

This feels like the disabling HPET thread all over again lmao

I had some stuttering with Deadlink on Steam, used DXVK for DX11 and it seemed better but I can't tell if it was just placebo or if it was actually working.
Steam overlay stopped working though, so I guess that's a sign?

 

I think if you use dxvk, Steam overlay doesn't work anymore. As for dxvk, is not a placebo, it helps a lot in some UE4 games, but some can't be fixed due to asset loading stutter that happens no matter how fast your CPU or storage is.

 

Good vid.

 

It's that or old school load screens which are much worse .

 

I actually prefer old school load screens. E.g. In DOOM 2016 they break up the levels via loadscreens but I find that less immersion breaking and less jarring for whatever reason than a hard traveral/asset stream hitch.

Maybe a good way to do it in between is the ole "corridor" / wait between two doors or in an elevator for the assets to load. I will personally take pretty much anything over a stutter. What I find really crazy is we've got full blown open world games out there like Arkham City/Knight that exhibit traversal stutter with uncapped framerates even on high end hardware -- how do you even ship something like that?

Clearly high detail modern games can handle streaming without stutter/hitches even without direct storage because it's been done (e.g. post patch Cyberpunk / Red Dead 2). Ideally there would be some asynchronous way to stream assets where you don't block the current frame but rather just show a "dummy" asset til the asset is loaded in subsequent frames, that sort of thing. I do not mind RAGE 1 style fade in if it means zero stuttering.

 

So many games are so well optimized, on PC, this is what i admire the most, so there is no excuse for "traversal stutter" that mere dev incompetence and stupidity, there are lots of old and new games that simply don't do this, which just makes me angrier on the issue. Is not our systems or Windows, it's useless devs.