NVidia Anti-Aliasing Guide (updated)

Discussion in 'Videocards - NVIDIA GeForce Drivers Section' started by Cyberdyne, Jan 29, 2012.

  1. smueqas

    smueqas New Member

    Messages:
    1
    Likes Received:
    0
    GPU:
    gtx 1650 4gb
    The old thread is confusing as F.
    Thanks for the update! i vote for a sticky!
     
  2. GuyFawkesGaming

    GuyFawkesGaming Member

    Messages:
    27
    Likes Received:
    1
    GPU:
    EVGA GTX 980 4GB
    I'd like to mention that I got Battlefield 4's builtin MSAA enhanced with TrSSAA. The following images I took have all settings maxed except for post aa and resolutions scale. I also have MFAA turned on but DSR turned off so that the focus can be on MSAA and TrSSAA. You have to make sure you set behavior flags to 'none' or 'treat override as enhanced' with Profile Inspector.
    Just 4x MSAA:
    [​IMG]
    [​IMG]
    4x enhanced with 8x TrSSAA:
    [​IMG]
    [​IMG]
    I also tested SGSSAA but the results were poor. Both the quality and performance were low. Furthermore, 8x was unavailable since BF4 only goes up to 4x MSAA (I tried enhancing the 4x to an 8x or HSAA, but got no results).
    TrSSAA again:
    [​IMG]
    [​IMG]
    SGSSAA:
    [​IMG]
    [​IMG]
    There's also an old anandtech thread here and even a guru3d one that both accurately say the same things about TrSSAA and SGSSAA when used in Battlefield 3 (TrSSAA good, SGSSAA bad). I'm gonna go out on a limb here and say this probably will work with other Frostbite engine games that support traditional aa types, such as Medal of Honor: Warfighter. There is also a reddit post (as well as a battlelog post that EA appears to have deleted in their spring cleaning, thanks EA!) that seems to imply that Battlefield 1 supports MSAA, although the setting is hidden. That means there's a good chance it's also enhanceable. Unfortunately I don't own BF1 so I can't test it.
     
  3. MrBonk

    MrBonk Ancient Guru

    Messages:
    3,237
    Likes Received:
    194
    GPU:
    MSI RTX 2080
    I'll add that. TrSSAA is probably worthwhile. Though if MSAA support is like in Bad Company 2 there are the occasional white line artifacts (And the MSAA isn't very high quality either) and i'm assuming enhanced SGSSAA won't look any good with other FB engine games either.

    Best looking solution would probably be 2.25xDSR+ 2xMSAA in game , enhance with 4xTrSSAA, turn on MFAA and then also turn on driver FXAA on top(very beneficial when downsampling with MFAA) if you have the performance to spare.
     
  4. Gonzo32

    Gonzo32 Member

    Messages:
    14
    Likes Received:
    1
    GPU:
    Rtx2080ti
    Hi guys, I am playing Mass Effect Legendary Edition and even tho they say that AA is improved, there are still lots of jagged edges, even at 4k and higher resolutions with supersampling. Any ideas about what to do? Are there any flags for it that could still work?
     

  5. MrBonk

    MrBonk Ancient Guru

    Messages:
    3,237
    Likes Received:
    194
    GPU:
    MSI RTX 2080
    If it's been upgraded to run on DX11 you are SOL outside additional super sampling.
    From the screens i've seen there's still a lot of aliasing. But if their TAA is unique and not based off of UE4 TAA, i'd honestly rather take a more consistently aliased image than the soupy artifact mess that UE4 TAA often is.

    You may have some luck by doing just slight downsampling and enabling in game TAA and then use Reshade or Profile Inspector to turn something like FXAA on , on top. Which would help when downsampling to tackle a lot of bad edge gradients that the in game AA leaves behind. It's the same reason why some games offer a TAA+FXAA option these days. Problem is really that it doesn't look that great at native resolution or lower resolutions in general.
     
  6. Gonzo32

    Gonzo32 Member

    Messages:
    14
    Likes Received:
    1
    GPU:
    Rtx2080ti
    Don't you think that maybe there is still a flag that could be compatible like the old trilogy had? nvidia inspector has anti aliasing compatibility for dx11 as well. Excuse my ignorance if I am saying something wrong, i am not very knowledgeable in this matter
     
  7. MrBonk

    MrBonk Ancient Guru

    Messages:
    3,237
    Likes Received:
    194
    GPU:
    MSI RTX 2080
    There is extremely basic compatibility for DX1x (Read: DX10) that was never fleshed out. Even the games it has flags for don't really do much of anything useful. (The one function that actually enables proper SGSSAA support doesn't exist in the driver for DX10. Without that one bit set, 99% of DX9 games would not work with SGSSAA)
    We had a petition on Nvidia's forums years and years ago asking for DX11 functionality. And they basically ended up nope'ing out of that.
    Instead they started their usual cycle of marketing bs introducing something new and useful and then abandoning it. (TXAA,HBAO/HBAO+ Driver support,DSR,MFAA,Gameworks,etc.)
     
    Markie and Cave Waverider like this.
  8. Gonzo32

    Gonzo32 Member

    Messages:
    14
    Likes Received:
    1
    GPU:
    Rtx2080ti
    Thank you as always for replying and helping me. Its a shame man, I am now with ME2 and this one has even more jaggies. I dont understand why DSR doesnt help at all, I guess its because aliasing comes from specular lighting? Im not sure. Reshade does help a bit but it doesnt do enough.
     
  9. RealNC

    RealNC Ancient Guru

    Messages:
    3,366
    Likes Received:
    1,566
    GPU:
    EVGA GTX 980 Ti FTW
    In my experience, DSR only helps at 4x. Anything else is meh. 4x with 0% smoothing is the only way I can get any proper results.
     
    Markie likes this.
  10. Markie

    Markie Active Member

    Messages:
    51
    Likes Received:
    1
    GPU:
    GTX 950 2Gb
    That's why I use custom resolutions instead. 2.25x works great that way, kind of a performance/IQ middle ground between native res and 4x.
     

  11. MrBonk

    MrBonk Ancient Guru

    Messages:
    3,237
    Likes Received:
    194
    GPU:
    MSI RTX 2080
    You have to think of it in simple terms. DSR is the simplest form of SSAA. It doesn't do anything fancy.
    X.y pixels sampled in a grid in: 1 pixel out (Insert resolve function here). At higher internal resolution it's still only sampled 1x per pixel and even at 4x resolution it's not enough sub pixel precision to resolve complex problems like shader/specular aliasing.
    (Which themselves are already sample starved if the developer doesn't implement some kind of specular anti aliasing at the texture level or mip level like CLEAN Mapping and variants. Too much high frequency information present in textures/normal maps = aliasing. See: Final Fantasy XIV, Phantasy Star Online 2 as DX9 examples on how not to handle textures https://blog.selfshadow.com/2011/07/22/specular-showdown/ And a good read on undersampling, why a good resolve is important , specular lighting etc in this presentation http://advances.realtimerendering.com/s2015/rad_siggraph_advances_2015.pptx
    Sadly a good resolve gets abused by bad TAA techniques like UE4 into a smeared mess)

    You are in essence very inefficiently just taking your 1x undersampled display resolution and making the pixels smaller and interpolating (Resolve) the 1 output pixel from the input samples.
    That's why a hybrid approach is usually needed for best results/performance. Because you'd need an absurd amount of oversampling with OGSSAA to accomplish what SGSSAA can do with fewer samples. Or even Temporal Anti Aliasing. (Sans artifacts. If it wasn't for the artifacts I'd love TAA. >_>) It's just not an efficient way to oversample and try to compute the ground truth.

    It takes something on the order of 16xOGSSAA (With a decent resampling function and resolve stage. Box/Bilinear fall apart at high ratios) to approach the quality of 8xSGSSAA temporally. 4x is not enough.
    I'd like to gets some videos maybe later but here's some screens of Devil May Cry 4. Running this game at 8k (With DSR Registry edit) produces incredible image quality and is not only on par with 8XSGSSAA temporally but has the added benefits of higher precision lighting/post process effects and better resolved textures. (Since the miplevels being used are much higher at such a resolution) But on an RTX 2080 rendering at 8k uses more than 2x the GPU that 8xSGSSAA does at 1080p during gameplay. And during cutscenes drops under 60FPS with a lot of effects going. Doesn't happen with 8xSGSSAA. 8xSGSSAA doesn't get everything here and the box resolve hurts it. But it's nearly 100% temporally stable, specular issues are non existent.
    Running the game at 4xOGSSAA looks decent, but there are not enough samples to resolve the image spatially and temporally.
    https://www.screenshotcomparison.com/comparison/14131 (Using HDR=HIGH improves 8xSGSSAA slightly but breaks DOF.)
    https://www.screenshotcomparison.com/comparison/14132 (Showing why a good resolve is important. Bicubic isn't the highest quality but Bilinear falls apart. Look at any high contrast edge of any kind and you'll see the missing values to make the transitions correct, resulting in broken lines and aliasing.)
    https://www.screenshotcomparison.com/comparison/14133


    Not to mention like with Specular mapping if a game further undersamples any part of the render path it's going to make it even harder to fix. Think the recent lower res specular pass on Rise of The Tomb Raider, or games that make heavy use of Dithering (Practically everything in UE4, like shadows,motion blur,grass,SSAO,hair) like in RE-Engine games. In cases like Resident Evil 2 and 3, the dithering of the hair can also greatly affect volumetric lighting when intersected depending on the scene and even with an absurd amount of sampling the hair can still be aliased. https://www.screenshotcomparison.com/comparison/14130
    For example:
    Here's a screen of RE3 using 3.24xDSR + TAA/FXAA and 4x SSAA enabled at the same time. (So in essence TAA/FXAA running at 6912x3888 with 2 stage downsampling. See below about RE Engine SSAA)
    https://u.cubeupload.com/MrBonk/re320200411235425309.png

    DSR is actually for what it is, very good compared to the in game Super Sampling modes most DX11 games offer these days because the resolve function is Gaussian (See animated example in the linked PPTX above on slide 22 and you can see why in game SSAA look poor) and the width of it is adjustable to the end user to their liking.
    Most games internal way of doing it use an extremely basic resolve that actually results in poor anti aliasing. (UE4 is another good example. The resolution scaling is terrible)
    The most recent example is Resident Evil VIII, which has some changes to the TAA that results in poorer handling of specular aliasing. Use DSR at around 25% smoothness at 4x res and it does a decent job helping tackle those with TAA enabled. Not perfect but big improvement. DSR's resolve function allows it to properly down filter the result without overshoot.
    Use the in game SSAA set to 200%(which is the same 4x res) and the specular aliasing is actually worse than without SSAA and a lot more distracting.



    TL DR
    This is why I always advocate for hybrid solutions and why using something like DSR (I recommend 25% smoothness personally. But I wouldn't go lower than 10% At 0% you add aliasing back into the image) in combination with other techniques gives the best results. (And not in game super sampling options) Especially TAA.
    None of them bar SGSSAA are adequate enough alone. Combine them and you can get great results. And it's sad more developers haven' taken this into account with their TAA solutions and instead just lazily slap on a badly written OGSSAA mode and call it a day. I can only imagine how nice a true super sampled/TAA hybrid could look.


    ALSO: I do own a few shares of Nvidia, if I ever got rich enough to buy a significant portion. I would try to weasel my way in and get someone to figure out how to add AA support to the driver for DX11...!
    One can dream.
     
    Last edited: May 18, 2021
    Gonzo32 likes this.
  12. Gonzo32

    Gonzo32 Member

    Messages:
    14
    Likes Received:
    1
    GPU:
    Rtx2080ti
    Thank you for sharing your knowledge.

    I wish I was one of those players who don't give a crap about aliasing. But once I see it while playing, it triggers my ocd and it drives me crazy. Its a shame that Nvidia doesnt support AA to DX11 games....it would be a life savior for some of us
     
    Cave Waverider likes this.
  13. KainXVIII

    KainXVIII Active Member

    Messages:
    87
    Likes Received:
    2
    GPU:
    MSI GTX 970
    I'm bumping 2 years old post :p Anything for Psycho?
    Even if PCGW page says
     
  14. Gonzo32

    Gonzo32 Member

    Messages:
    14
    Likes Received:
    1
    GPU:
    Rtx2080ti
    The new expansion for Elite Dangerous just released and the jaggies in the video are awful lol, I guess there is no fix for this probably

     
  15. RealNC

    RealNC Ancient Guru

    Messages:
    3,366
    Likes Received:
    1,566
    GPU:
    EVGA GTX 980 Ti FTW
    Yep. It's half-assed. Just like everything else Frontier is doing with this game.
     
    Gonzo32 likes this.

  16. MrBonk

    MrBonk Ancient Guru

    Messages:
    3,237
    Likes Received:
    194
    GPU:
    MSI RTX 2080
    Speaking of half assed, so is Phantasy Star Online 2 NGS.
    Good god the image quality is so bad. The TAA doesn't have all the smearing artifacts (Just some minor ghosting) of other TAA but it almost doesn't do anything for the absurd amount of aliasing everywhere present. (And if this is the TAA Base PSO2 is getting then lord help me when they gut the DX9 renderer out like a fish and this is all i'm left with. In addition to things that were perfect transparencies before now being a dithered mess. On top of the fact users without an AVX supported CPU are now SOL. Which is stupid) There's an option called "Specular Anti Aliasing" But it's not clear what it actually does. (Reflections maybe?) It certainly doesn't do anyting for *actual* specular aliasing.

    The texture streaming and LOD management(And draw distance), even on an NVME drive is hot garbage based on how the benchmark looks. It's even inconsistent between runs where one run an asset will be fully loaded and another run it's potato soup.Also, I don't think i've seen so much dithering in my life in a current gen game...
    In order to get even decent IQ you almost have to downsample+TAA/FXAA. But the performance at 4k on a 2080 is not great. Using dynamic res is hyper aggressive (Even when capped to 60 and it hits 60), but still doesn't kick in enough when it drops below 60. But you have to because there are so many visual discontinuities from the low quality rendering going on everywhere that it looks extremely bad without it. The screen space shadows in particular are super low quality and come with all the same draw backs as SSR/SSAO as objects occlude one another they draw in and out. Flicker like crazy.

    Some screens and comparisons:
    https://www.screenshotcomparison.com/comparison/14338
    https://www.screenshotcomparison.com/comparison/14339 (The dithering doesn't go away. At 1:1 scale at 4k it's even noticeably ugly)
    https://www.screenshotcomparison.com/comparison/14341
    https://www.screenshotcomparison.com/comparison/14342 (Look at the tree to the left right corner. LOD fail)
    Without the obnoxious DOF blurring out the background, the lighting in general looks very flat as a lot of open world games struggling for performance look these days.
    https://u.cubeupload.com/MrBonk/pso22021052516110184.png
    https://u.cubeupload.com/MrBonk/pso22021052515573418.png (Look at how even more flat it looks without screen space shadows. There's so few actual shadow maps)
    https://u.cubeupload.com/MrBonk/pso22021052516110184.png
    https://u.cubeupload.com/MrBonk/pso22021052516111015.png (If I remember right this is without SSS and SSAO.)
    https://u.cubeupload.com/MrBonk/pso22021052516110496.png (Awful distance LOD and grass draw distance only like 20-30 feet away)
    The TAA is so bad at figuring out sub pixel detail it makes my character look straight up cross eyed from a distance. https://u.cubeupload.com/MrBonk/pso20210525194035018.png (lol)
    The Screen Space Shadows can even bleed through smoke... https://u.cubeupload.com/MrBonk/pso20210525194230027.png



    I really wish they had just kept it instanced like PSO1/2 but make the areas less corridor like and more wide like Crysis instead of this. And the character models by default are so damned derpy looking compared to base PSO2.
    After like 2 hours of tweaks this is the closest I could get my existing PSO2 character face to match.
    https://www.screenshotcomparison.com/comparison/14340
    I already had low expectations until I saw it with my own eyes, but it's as bad as I feared when they first showed it off.

    While base PSO2 is certainly not the most advanced looking game on a technical level, to be fair it looked visually outdated even in 2012. But at least image quality isn't complete garbage as long as you have a GPU powerful enough to either Downsample or use 8xSGSSAA. Even without forcing AA and all you have is the in game FXAA IQ is better than NGS.
    https://u.cubeupload.com/MrBonk/pso20210510034415007.png
    https://u.cubeupload.com/MrBonk/pso20210511193942001.png
    https://u.cubeupload.com/MrBonk/pso20210511213322009.png
    https://u.cubeupload.com/MrBonk/pso20210512225400003.png
    https://u.cubeupload.com/MrBonk/pso20210514235250022.png
    https://u.cubeupload.com/MrBonk/pso20210518181509002.png
    https://u.cubeupload.com/MrBonk/pso20210518182112007.png
    https://u.cubeupload.com/MrBonk/pso20210518231649014.png
    https://u.cubeupload.com/MrBonk/pso20210519001842029.png

    Even FFXIV which is 8 years old now looks better than NGS.
    NGS distance rendering https://u.cubeupload.com/MrBonk/pso20210525194235029.png
    FFXIV (Panorama made 5+ years ago from 1080p 4xSGSSAA screenshots) https://u.cubeupload.com/MrBonk/623panocopy.png
     
    Last edited: May 26, 2021
    KainXVIII and Gonzo32 like this.
  17. MrBonk

    MrBonk Ancient Guru

    Messages:
    3,237
    Likes Received:
    194
    GPU:
    MSI RTX 2080
    Err I could've sword Psychonauts had an entry lol.

    Is it actually DX9? Tried without a flag? If doesn't work, try 0x00000041
     
    KainXVIII likes this.
  18. KainXVIII

    KainXVIII Active Member

    Messages:
    87
    Likes Received:
    2
    GPU:
    MSI GTX 970
  19. MrBonk

    MrBonk Ancient Guru

    Messages:
    3,237
    Likes Received:
    194
    GPU:
    MSI RTX 2080
    Awesome, i'll add it. Thanks for checking it out!
     
    KainXVIII likes this.
  20. Gonzo32

    Gonzo32 Member

    Messages:
    14
    Likes Received:
    1
    GPU:
    Rtx2080ti
    Quick question, can someone tell me what are the best settings for SMAA when using reshade? I noticed there are a lot of different parameters in the reshade menu. Any recommendation for max efficiency?

    Thank you
     

Share This Page