NVidia Anti-Aliasing Guide (updated)

Discussion in 'Videocards - NVIDIA GeForce Drivers Section' started by Cyberdyne, Jan 29, 2012.

  1. MrBonk

    MrBonk Guest

    Messages:
    3,385
    Likes Received:
    283
    GPU:
    Gigabyte 3080 Ti
    Halo 2? It was a port that required vista so I have always called it H2Vista. And NOW I realize I was replying to the guy talking about AA for Halo 2 and not you. Sorry!

    For your example since it is DX11, you can't force AA. So you are left with very few options. Post Process AA like FXAA/SMAA or that+Downsampling. But as you said downsampling is too taxing for your scenario. So you don't have really any options sorry. I don't know what ratio you have tried but if you tried 4x and it's too draining on performance. Try 2x/2.25x.


    Definitely. I prefer to use LumaSharpen most often myself and I use it often.
    Especially with a lot of modern games that use extremely aggressive TAA. As even with downsampling it can still be too aggressive. (In some examples like in certain UE4 builds the TAA goes from being sharp when there is little to no movement but as soon as movement does occur. It turns into smudge city. Leading to this weird constant shift from blurry to sharp. Another example of this is the TAA in Hitman 2)

    (Even with in game sharpening. Like with Resident Evil 2 Remake. Their built in sharpening barely brings the image back to passable looking. And without it it's a smudgy mess. But it recovers enough detail that you shouldn't disable it if you want to sharpen the image manually. I experimented with sharpening RE2 with the built in one disabled and the results were never adequate enough. Especially with adaptive sharpen. As it still was a smudgy mess afterwards. So you essentially have 2 pass sharpening in this case. But with moderation you will never know the difference otherwise.)

    You are wasting your performance if you use 0% in my opinion. It just leads to adding aliasing back into the image. Which somewhat defeats the point of AA in the first place.
    Nvidia just gives you the ability to tweak the resolve function width to your taste. 0% reintroduces aliasing into the image because the filter isn't wide enough to properly resolve the information. Which is great if you are going for pixel art upscaling/downscaling. But not Anti Aliasing.

    In order to get the output from the input, the resolve filter has to determine how much to average between samples. If it's too wide, then too much information can be lost and the output is softer. (http://screenshotcomparison.com/comparison/140) If it's too narrow then it can't get enough information to filter between points. Resulting in aliasing. http://screenshotcomparison.com/comparison/141 And the filter function you use for this can greatly impact the output.

    Here's an example of what happens (In an extreme example) when the function isn't sufficiently sampling enough around points to properly resolve the image. The lower it is the more aliasing is introduced into the image. (In this example, the function starts to break below 30%)
    http://screenshotcomparison.com/comparison/138
    Another example of what 0% will do with an easy to see comparison using text. With 4 samples per pixel the text should be completely smooth. 0% cannot resolve this properly http://screenshotcomparison.com/comparison/139
    Here's a 3D example that should be aliasing free. 4xDSR (4xSSAA) + 8xMSAA with a Gaussian resolve + Temporal AA with 0% DSR vs 25% DSR http://screenshotcomparison.com/comparison/142 0% cannot resolve edges of these polygons correctly. Here's the source image https://abload.de/img/msaafilter_2019_05_16gbkyo.png and as you can see that when viewed 1:1 it is essentially aliasing free along the same area.
    There's a reason why the default value of 33% was chosen. This is the value that gets the closest middle ground of properly resolving these high contrast edges vs image sharpness. I prefer 25% and even 25% still leaves visible stepping compared to 33%

    You can use 0% if you want because it doesn't look totally awful. (It still looks better than no AA but you are spending all that performance to take 4 steps forward and 2 steps back)
     
    Last edited: May 17, 2019
    Anontsuki likes this.
  2. MrBonk

    MrBonk Guest

    Messages:
    3,385
    Likes Received:
    283
    GPU:
    Gigabyte 3080 Ti
    4xSGSSAA will get a lot of the aliasing in ME. But it won't get all of it, so you will still see some. (You can also change that last 5 to a 1 and see if that makes some of the aliasing better. Sometimes this 5 results in a sharper image, but with more aliasing left unresolved)
    You shouldn't set in the in game MSAA if you are forcing it.
    Set it to Override and then 4xM/4xSG.
    You can Enhance it if you want but the results will tend to vary significantly from forcing it. Often worse.
     
    Tizoc789 likes this.
  3. Anontsuki

    Anontsuki Member

    Messages:
    16
    Likes Received:
    1
    GPU:
    GTX 1070 MSI Gaming
    Thanks for the writeup, MrBonk! =)
     
  4. GuyFawkesGaming

    GuyFawkesGaming Member

    Messages:
    28
    Likes Received:
    2
    GPU:
    EVGA GTX 980 4GB
    For Halo2Vista I can confirm an improvement to enhanced SGSSAA (enhance + 4x MSAA + 4x SGSSAA) over builtin AA, though I haven't tested it compared to plain forced 8x MSAA.
    @PoeticSanity you are in luck as Rising Storm 2 was on sale so I bought it. DSR does in fact improve aliasing, though only in fullscreen mode, not boarderless. I have a similar issue with BioShock Remastered when I tried to force boarderless plus the issue I requested help with a while back with Red Orchestra 2 and Chivalry: Medieval Warfare when using SGSSAA and boarderless. I'm just gonna say it's safe to assume Unreal Engine doesn't like AA + boarderless.
    Also I bought the original Red Orchestra: Red Orchestra Ostfront 41-45. Killing Floor 1's 0x000000C1 flag works well on it. didn't test out all the rendering modes but considering that it was made by the same devs as Killing Floor, it likely uses the same version/build of the Unreal Engine.
     

  5. MrBonk

    MrBonk Guest

    Messages:
    3,385
    Likes Received:
    283
    GPU:
    Gigabyte 3080 Ti
    That sounds about right.
     
  6. Tizoc789

    Tizoc789 Guest

    Messages:
    31
    Likes Received:
    3
    GPU:
    GTX 970M
    Alright thanks, I'll test that out now.

    EDIT: OK I seem to have gotten it resolved.
     
    Last edited: May 20, 2019
  7. ahmlet

    ahmlet Member

    Messages:
    40
    Likes Received:
    0
    GPU:
    980 ti g1gaming
    How to add AA to Doom 3? totally forgot the settings and cant find them
     
  8. MrBonk

    MrBonk Guest

    Messages:
    3,385
    Likes Received:
    283
    GPU:
    Gigabyte 3080 Ti
    If using the original Doom 3. You don't need any flags or anything. Just set to Override and set MSAA and "Transparency Super Sampling" for SGSSAA.
    Do note though that there is a bug with the drivers introduced a while back that causes some visual glitches with textures in OGL games and SGSSAA.
    However in Doom 3 they only manifest on certain specific surfaces with alpha tested textures or visual displays at very specific viewing angles.
    I had to look for it purposefully to notice something not looking right when I played Doom 3 a few months ago. It's not very noticeable.

    Also: If you plan on using mods, be very careful about what post processing effects you use as it will be affected by forcing SGSSAA and can lead to a very blurry image if you aren't careful.
    If you use a mod that has Sikkpin's DoF shader, 8xSGSSAA breaks the DoF and causes aliasing as well. 4x is unaffected and the image is stable.
    Downsampling can help a lot too.
     
  9. Postal-Dude

    Postal-Dude Guest

    Hm. I don't seem to be able to activate SGSSAA in Chronicles Of Riddick - Assault On Dark Athena. I followed the instructions to the letter. I also disabled the prevent anti aliasing flag / 0x20992431, as it was called once. I also tried to activate SGSSAA alongside ingame MSAA. But absolutely nothing has an effect. I dont even get normal Multisampling to work.

    Can anyone confirm this with latest Nvidia driver? I am not sure if the problem is on my end or if it is just not working for anyone anymore. Possible fix? Compatibility bits?
     
  10. MrBonk

    MrBonk Guest

    Messages:
    3,385
    Likes Received:
    283
    GPU:
    Gigabyte 3080 Ti
    i'll have to download it and give it a shot when I have time this week maybe.
     
    Postal-Dude likes this.

  11. stillmehungry

    stillmehungry Guest

    Messages:
    2
    Likes Received:
    0
    GPU:
    780 lightning
    Anyone know of any fix for AA In Dayz Standalone. The edges of fences and rooftops are awful for me. I've played around with profile inspector but can't seem to find something that works. Appreciate any help I can get
     
  12. MrBonk

    MrBonk Guest

    Messages:
    3,385
    Likes Received:
    283
    GPU:
    Gigabyte 3080 Ti
    Pretty sure it's Dx11. So no.
     
  13. gaintbeam

    gaintbeam Guest

    Try TrSSAA? There’s another thread that gives such instructions. Problem is once you get supersampling going you can’t enable SSAO it seems. But with ENB and Reshade it might work.
     
  14. GuyFawkesGaming

    GuyFawkesGaming Member

    Messages:
    28
    Likes Received:
    2
    GPU:
    EVGA GTX 980 4GB
    I'm gonna do an update to my tests on Halo 2. Using any form of Override has no effect on the game (probably due to no bits), Enhance + TrSSAA give best results, Enhance + SGSSAA technically works but corrupts some textures making them really blurry and looking like you're driving by them really fast. The texture corruption can best be seen on the map blood gulch where the grass is just completely smeared. Not sure if this is useful for your master list but wanted to bring closure to my question.
     
  15. Astyanax

    Astyanax Ancient Guru

    Messages:
    17,011
    Likes Received:
    7,352
    GPU:
    GTX 1080ti
    halo 2 uses a form of rendering that cannot have any AA no matter what you try to do.
     

  16. MrBonk

    MrBonk Guest

    Messages:
    3,385
    Likes Received:
    283
    GPU:
    Gigabyte 3080 Ti
    It does have native AA built into it though. And as noted you can enhance it with TrSSAA. Which helps with all the trees and foliage in the game.


    Also: I just tried RTX Quake 2 and it's about as poor IQ wise as I expected. The Temporal de-noiser+TAA is super blurry and has lots of side effects. Imagine standard TAA problems and add a bunch more problems on top. All the while eating of 20+% of the GPU !
    Using 150% resolution scale improves it slightly better but the denoising can't keep up.(Especially under water) And at this point with medium GI an RTX 2080 is only running at 30FPS (1080p base. Assuming correct scale. This means 1620p rendering).If the denoiser is off, then with 150% resolution it can hit in the neighborhood of 50+FPS.

    The lighting is nice and all but good lord is it not worth the performance. And the image quality is very low as a result. I feel like with current rasterization techniques they can get something that approximately looks almost as good but with way better image quality and performance. I feel like using ray tracing would be better used to come up with some better way to feed into TAA to improve the quality to fix occlusion artifacts and ghosting. Like casting rays to see where what pixel is behind what object.

    I will say the ray traced water looks way better than any screen space reflection shader i've seen but it still comes with artifacts from the TAA/T-Denoising.

    If it ran better, then it would look a lot better. At 4k it runs at about 20-25FPS. But a lot of the artifacts from the denoising and TAA become a lot less visible at 1080p.

    So if we had the power to run this at 4x+ of your native resolution at 60FPS I'd say that would be perfect. But that's only for a game as simplistic as this. For modern games it will never happen. At factors over 4x from native the difference between the reference render and real time becomes less apparent.

    Examples:
    Fine 1 frame under water. http://u.cubeupload.com/MrBonk/q2rtx201906061825495.jpg
    De-noiser is confused next frame (this pops up several times a second) This is running at 150% resolution too http://u.cubeupload.com/MrBonk/q2rtx201906061825505.jpg

    100% resolution http://u.cubeupload.com/MrBonk/100percent.jpg
    150% resolution http://u.cubeupload.com/MrBonk/150percent.jpg
    100% resolution reference render (hit pause break) http://u.cubeupload.com/MrBonk/100percentreference.jpg
    Exaggerated sharpening of motion http://u.cubeupload.com/MrBonk/exaggeratedsharpen10.jpg notice the low frequency areas, inconsistency in the ground texturing. There is 1 area where the texturing looks as it should (to the left slightly) , the denoising is failing on the periphery of the image (incomplete resolve) and then typical TAA smudging around the pistol and ground texture around it and then the slightly more complete resolve on the lower right hand side http://u.cubeupload.com/MrBonk/exaggeratedsharpen10.jpg

     
    Last edited: Jun 7, 2019
  17. GuyFawkesGaming

    GuyFawkesGaming Member

    Messages:
    28
    Likes Received:
    2
    GPU:
    EVGA GTX 980 4GB
    Since I'm new to the enhance method and use to the override, I hope you don't mind me asking 2 questions about it. Does it still need a depth bit for reshade or are compatibility bits completely out of the question? And can you enhance a app's builtin low/limited MSAA setting to a high one (Halo 2 uses 4x max built in, can I enhance to 8x)?
     
  18. MrBonk

    MrBonk Guest

    Messages:
    3,385
    Likes Received:
    283
    GPU:
    Gigabyte 3080 Ti
    I actually haven't tried with Reshade and Enhancing so you'll have to find out. And that depends on whether a specific game's SSAA/MSAA breaks depth detection or not depending on the version of Reshade you use.

    Sometimes enhancing to a higher level works but I don't remember specific examples. More often than not you can enhance the TrSSAA/SGSSAA to a higher level of MSAA than supported by the game. With TrSSAA this works just fine. With SGSSAA results may not be satisfactory.

    I updated the spreadsheet to show that enhancing trssaa works with H2V
     
  19. GuyFawkesGaming

    GuyFawkesGaming Member

    Messages:
    28
    Likes Received:
    2
    GPU:
    EVGA GTX 980 4GB
    I did more testing, this time with TrSSAA off. The depth bit and AAfix some games need has no effect on H2V. Also It's 4x MSAA is indeed enhanceable to 8x, best seen on first person weapon models. I also did try enhancing the builtin 4x AA with 4x from the inspector (assuming Nvidia's would be more "optimized") and saw no visible difference. Lastly, I got HBAO sort of working using the flag from Halo CE on your other guide, I'll save the details for that thread though.

    EDIT: just tested a bit more just now, MSAA also enhances up to settings like 4x4, 8xSQ and 32xS (I think they're called "OGSSAA" but not sure because I never used them before) both with and without TrSSAA.
     
    Last edited: Jun 8, 2019
  20. Exaercase

    Exaercase Guest

    Messages:
    3
    Likes Received:
    0
    GPU:
    GT 740 - 2048
    Hello,

    I'm sorry if entry exist but i don't find any info on AA Flags for Assassin Creed Rogue.

    Someone Could Help me
    Thx in advance
    Best Regard, ExA
     

Share This Page