NVidia Anti-Aliasing Guide (updated)

Discussion in 'Videocards - NVIDIA GeForce Drivers Section' started by Cyberdyne, Jan 29, 2012.

  1. Markie

    Markie Member Guru

    Messages:
    114
    Likes Received:
    26
    GPU:
    GTX 950 2Gb
    The AxB in OGSSAA means horizontal x vertical samples. 2x1 samples pixels twice horizontally and once vertically with 1x2 being the opposite, and only horizontal or vertical edge aliasing should be treated depending on which one you choose. I believe 2x1 should usually be more demanding than 1x2 as there's more horizontal than vertical pixels in most resolutions and aspect ratios.
     
    Klemc likes this.
  2. MrBonk

    MrBonk Guest

    Messages:
    3,385
    Likes Received:
    283
    GPU:
    Gigabyte 3080 Ti
    Ok, here are a few new additions. (Lot of Unity Engine)

    Lumo - This game is sadly impossible to force AA in without problems. You can kind of get it to work by disabling certain effects but it still has problems. Oddity for a Unity game.

    Iris.Fall - 0x004000C1 - SGSSAA,OGSSAA,HSSAA(With SGSSAA) Negative Lod bias can be helpful.
    NoAA https://u.cubeupload.com/MrBonk/1aNoAA.png
    8xSGSSAA https://u.cubeupload.com/MrBonk/d188xSG.png
    8xSGSSAA -1.25 LOD https://u.cubeupload.com/MrBonk/8xSg15LOD.png
    4x4 OGSSAA https://u.cubeupload.com/MrBonk/e4x4.png

    Plight of The Zombie - 0x004000C1 - MSAA, SGSSAA,OGSSAA,HSSAA
    No AA https://u.cubeupload.com/MrBonk/potz2022040417521438.png
    8xMSAA https://u.cubeupload.com/MrBonk/potz2022040417581694.png
    8xSGSSAA https://u.cubeupload.com/MrBonk/potz2022040417585248.png
    4x4 https://u.cubeupload.com/MrBonk/potz2022040418004164.png

    ExZeus 2 (Legacy) - MSAA, SGSSAA,OGSSAA,HSSAA - AA fix is useful to reduce softness with SGSSAA
    No AA https://u.cubeupload.com/MrBonk/exzeus22022040401153.png
    8xMSAA https://u.cubeupload.com/MrBonk/exzeus22022040401265.png
    8xSGSSAA https://u.cubeupload.com/MrBonk/exzeus22022040401172.png
    4x4 https://u.cubeupload.com/MrBonk/exzeus22022040401274.png

    RaidersSphere4th - MSAA,SGSSAA,OGSSAA,HSAA(+-SGSSAA) - AA fix can improve some texture aliasing.
    This must be used with the Shader level 1 below maximum. It does some funky stuff and also seems to do some basic SSAA. Which looks decent except the texture aliasing is still pretty bad. You can force AA in this mode but performance tanks. Performance in this game is already a bit all over the place. But it's a small Doujin game so I can't fault it. The game was designed for 4:3 though, which leads to these odd borders at the sides in 3rd person view like one of the rendering effects hasn't be adjusted for it sadly.
    No AA https://u.cubeupload.com/MrBonk/b83noaa.png
    8xMSAA https://u.cubeupload.com/MrBonk/9118xMSAA.png
    8xSGSSAA https://u.cubeupload.com/MrBonk/a58xSGaafix.png
    8xSGSSAA with max shader setting https://u.cubeupload.com/MrBonk/Shaderhigh8xSG084012.png
    4x4 https://u.cubeupload.com/MrBonk/dc74x4.png

    AI: The Somnium Files - You can enhance the in game MSAA with MFAA,TrSSAA or SGSSAA. Like Zero Time Dilemma this can introduce some softness so minor sharpening may be needed depending on your preferences.
    One of few Unity games that support MSAA (Thank you Kotaro Uchikoshi's team!). You can get really flexible to get great image quality out of this by doing Hybrid Sampling. (IE: Downsampling + 2xMSAA + 2xSGSSAA + MFAA + FXAA).
    No AA https://u.cubeupload.com/MrBonk/AITheSomniumFiles202.png
    8xMSAA https://u.cubeupload.com/MrBonk/efeAITheSomniumFiles202.png
    8xEnhanced SGSSAA https://u.cubeupload.com/MrBonk/188AITheSomniumFiles202.png
    5.7k Res with 4xSGSSAA downsampled to 4k https://u.cubeupload.com/MrBonk/57k4xSGdownto4k.jpg


    This brings the sheet to 754 rows harr harr harr!!
     
    KainXVIII and didamangi like this.
  3. Klemc

    Klemc Master Guru

    Messages:
    308
    Likes Received:
    60
    GPU:
    4070-ti ARGB - PNY
    Thanks, that's well explained.
    I'll compare this two with current 8xQ applied, soon.

    BTW, can i ask a thing about VSync, because all i find about "1/2 refresh rate" option in NVPinspector is about 30FPS game beeing fixed,
    BUT, when i apply this to some games (example: Investigator), game runs good instead of slow and at 120hz. But does it halfes FPS ?! In that case i won't use it anymore !
    The 1/2 option from NVPI is about normal VSync, not Adapative like from NVControl panel ; if i didn't misread !!

    My GPU's an EVGA 3060 XC (i7 9700F).
     
  4. KainXVIII

    KainXVIII Member Guru

    Messages:
    106
    Likes Received:
    4
    GPU:
    MSI GTX 970
    Is there a working AA solution for Arx Fatalis (steam version without arx libertatis mod) that will not break text rendering?
     

  5. Klemc

    Klemc Master Guru

    Messages:
    308
    Likes Received:
    60
    GPU:
    4070-ti ARGB - PNY
    Blackwood Crossing - PCGWiki tells D3D11 but Reshade installs opengl DLL and loads...

    BTW, i can' t see anyhing different when applying FXAA/SMAA with reshade.

    This is DX11's Unity game, so, no flag working if i remember well.

    No solution ?..

    DSR makes the game lock/freeze/hang before going to 3840 (from 1440p screeen).
     
  6. KainXVIII

    KainXVIII Member Guru

    Messages:
    106
    Likes Received:
    4
    GPU:
    MSI GTX 970
    Oh, and 0x000010C1 SGSSAA bit for original COD: Modern Warfare does not work (anymore?), it causes serious graphical glitches.
     
  7. MrBonk

    MrBonk Guest

    Messages:
    3,385
    Likes Received:
    283
    GPU:
    Gigabyte 3080 Ti
    Really? That seems strange that they would've updated the game and broke it somehow in the last decade.
    I still have screenshots of it working on my hard drive but those are also, *edit* literally are 10 years old. ;The timestamps are from 2012 lol; (Not gonna lie, I pirated the game back then just to test it for stuff like that)
    Guess i'll have to look around for a cheap deal on it so I can check it out.

    EDIT: Just bought it on Steam and you are right it's all kinds of broken

    OK: The problem seems to be the Depth of Field. Disable it , no problems.
    If you want the DOF you can use 0x000012C1 instead and it works just fine.
    Also, if you enable Dual Video Cards option you can get a pretty decent FPS boost at lower resolutions even with AA enabled.

    If you want to Anti Alias the DOF you can use 0x000032C1 at the cost of some softness. At high res though this will tank performance. At 4k with 8xSGSSAA with a 3080 Ti just one scene in the tutorial map you are looking at 36FPS when ADS vs 66FPS when ADS with DOF enabled and 32C1.

    I have updated the document to reflect this.

    EDIT3:The game literally went on sale the morning after I bought it on steam to test. I've tested 2.1 hours so far. Activision is an awful company, so i'm going to refund it and rebuy it on sale. I'm a terrible person obviously.
     
    Last edited: Apr 7, 2022
    KainXVIII likes this.
  8. Klemc

    Klemc Master Guru

    Messages:
    308
    Likes Received:
    60
    GPU:
    4070-ti ARGB - PNY
    For blackwood Crossing, i found a way to disable DOF (allow_create_device=1 for GOG version instead of 2 !!)...

    ... Helix uses 3DMigoto, so i used the upscale ini settings and now it's OKay, better at least, but for a 1440p monitor, i set resollution to 8k !!!!

     
  9. Gonzo32

    Gonzo32 Member

    Messages:
    44
    Likes Received:
    8
    GPU:
    Rtx2080ti
    Hi there, one question: whats the best anti aliasing option while using reshade? specially for specular aliasing
     
  10. Klemc

    Klemc Master Guru

    Messages:
    308
    Likes Received:
    60
    GPU:
    4070-ti ARGB - PNY
    SMAA's not good if game uses MSAA.

    SMAA+FXAA in that order because if FXAA's loaded first it conflicts with SMAA.
     
    Last edited: Apr 15, 2022

  11. MrBonk

    MrBonk Guest

    Messages:
    3,385
    Likes Received:
    283
    GPU:
    Gigabyte 3080 Ti
    It depends on whether it's a DX9 game or not. If it's not a DX9 game, there's not a whole lot you can do about some specular aliasing. A combination of downsampling (DLDSR May improve how it looks.)+ something like FXAA(Or SMAA)&NFAA(Or DLAA)

    When I played Dead Space 2 last year I used 4xDSR+FXAA&NFAA. Which looked great most of the time.
    https://www.screenshotcomparison.com/comparison.php?id=13618
    https://www.screenshotcomparison.com/comparison.php?id=13619
    https://u.cubeupload.com/MrBonk/deadspace22021072122.jpg
    https://u.cubeupload.com/MrBonk/929deadspace22021072122.jpg (You can see to the middle left some leftover specular aliasing. Also around the fingers of the man grabbing Isaac and around the plasma cutter)
    https://u.cubeupload.com/MrBonk/deadspace22021072123.jpg
    https://u.cubeupload.com/MrBonk/deadspace22021072200.jpg
    https://u.cubeupload.com/MrBonk/deadspace22021072223.jpg
    https://u.cubeupload.com/MrBonk/deadspace22021072500.jpg
    There were definitely a lot of times where it was inferior to SGSSAA with some specular issues. But not much you can do about that. Even when games do include things for pre-filtering the specular component of lighting, undersampling is still a problem.
     
  12. Klemc

    Klemc Master Guru

    Messages:
    308
    Likes Received:
    60
    GPU:
    4070-ti ARGB - PNY
    Well, just to report something in relation with AA, i found a graphic glitch in Watch Dogs (but, tested only with the mod Remastered applied), the hack minigame interface (when hacking ctOS's places) does not show corectly at all, and becomes unplayable.

    That occurs when, like PCGWiki tells, game config xml file's manually edited, so, supersampling=1 instead of 0.

    BTW, thanks to the bits page, i applied ingame TXAA instead and with enhance SGSSAA it gives even better IQ than the ini SSA tweak anyway.

    RTX 3060 XC driver from this site
     
    MrBonk likes this.
  13. Markie

    Markie Member Guru

    Messages:
    114
    Likes Received:
    26
    GPU:
    GTX 950 2Gb
    Answering @Gonzo32 as well as going into a bit of a rant:

    The only post-processing AA solutions I use (either combined with downsampling or alone at native resolution when I don't have spare GPU power for SGSSAA/downsampling/MSAA) are Reshade's SMAA at the "ultra" quality settings (not sure if that actually was or still is an official thing but I found these values in a forum post and have been using them since forever, it's very effective and works good or great in pretty much any game) or NVCP FXAA which I prefer over Reshade's FXAA (not sure if it's possible to replicate Nvidia's implementation by tweaking the Reshade one - I will still look into it (any idea @MrBonk?)).

    These two options are better than in-game implementations 95% of the time, with rare exceptions being when the game's own post-processing AA actually affects geometry and effects in a special manner rather than just applying over the whole image equally. For example, no external AA cleans up edges as precisely as the in-game FXAA in Alan Wake, especially geometry that's behind volumetric lighting and high contrast edges. In Assassin's Creed Unity, the FXAA implementation is actually pretty good and reduces shimmering rather than just blurring it like NVCP FXAA does. In The Evil Within, the in-game AA solutions (FXAA, MLAA, SMAA) also seem to deal with geometry aliasing more accurately than any external AA and you're better off keeping it on even with external AA on top.

    Whether Reshade's SMAA or NVCP FXAA is better is situational. It really depends on how much aliasing there is, how prevalent shimmering is and if you're looking for a softer look (FXAA) or just want to clean up edges while blurring the image as little as possible (SMAA). Examples of different scenarios:
    1. In Dead Space (while already downsampling), NVCP FXAA is better than Reshade's SMAA to my eyes. That game (as well as the second one I think) has just way too much aliasing and some additional blur is welcome.
    2. In NFS: Hot Pursuit (while already downsampling), Reshade's SMAA is better than NVCP FXAA. It definitely cleans up edges better as well as keeping the image sharper.
    3. In games like Crysis 3, The Evil Within and Sniper Elite 4 for example, the in-game AA implementations are just too weak and fail at treating aliasing and shimmering well (but especially shimmering). In those cases, combining the in-game FXAA with NVCP FXAA and applying Reshade's Lumasharpen over it can give surprisingly good results for how performant it is and since it might not be possible for everyone to do downsampling or use the more expensive AA solution in these games (MSAA in Crysis 3 for example).
     
    Last edited: Apr 16, 2022
  14. Martigen

    Martigen Master Guru

    Messages:
    535
    Likes Received:
    254
    GPU:
    GTX 1080Ti SLI
    For those playing around with SMAA and FXAA in Reshade, perhaps you haven't seen this:

    https://reshade.me/forum/shader-presentation/7604-h-ybrid-high-q-uality-a-nti-a-liasing-hqaa

    It combines both, adds its own improved edge detection, and adds a host of optional features such as sharpening (that operates off the edges detected only), smoothing, temporal stabilizing, and a brilliant hysteresis reconstruction feature that restores text and other fine details that get smudged from FXAA/SMAA, and a whole lot more.
     
  15. Markie

    Markie Member Guru

    Messages:
    114
    Likes Received:
    26
    GPU:
    GTX 950 2Gb
    @MrBonk Is the Sonic Generations flag correct? Oh, and you still forgot to add HSAA to Worms Ultimate Mayhem when you fixed the stuff I mentioned. By the way, could you please check the HBAO+ thread? I found a flag for Dark Void and made a few comments about other games in the spreadsheet as well, I tagged you there but you haven't added the flag or answered yet...
     
    Last edited: Nov 14, 2022
    KainXVIII likes this.

  16. MrBonk

    MrBonk Guest

    Messages:
    3,385
    Likes Received:
    283
    GPU:
    Gigabyte 3080 Ti
    You can get decently results with Crysis 3 with the in game AA with downsampling at 60FPS on modern GPUs. If you want to target 30FPS you can get even better results with some trickery.
    I haven't tested it(Because I refuse to buy anything on EGS), but the remastered version has DLSS I guess. And I don't recall if they changed SMAAT2x to be more like modern TAA like in C1/2R.

    That flag was added after testing several years ago and was higher quality from what I recall. I'd be willing to retest it.
    I must have missed the HBAO+ information and worms. I'll take a look
     
    Markie likes this.
  17. Markie

    Markie Member Guru

    Messages:
    114
    Likes Received:
    26
    GPU:
    GTX 950 2Gb
    I went back and found that 0x084112C5 was the original flag, then someone came and claimed 0x004010C5 was less blurry, and later it was said it probably didn't work on aspect ratios such as 16:10 so it likely needed to be 0x084010C5.
     
    Last edited: Nov 14, 2022
  18. Gonzo32

    Gonzo32 Member

    Messages:
    44
    Likes Received:
    8
    GPU:
    Rtx2080ti
    Hi there, thank you for the information. Could you link me this tutorial to learn how to use smaa ultra?
     
  19. Markie

    Markie Member Guru

    Messages:
    114
    Likes Received:
    26
    GPU:
    GTX 950 2Gb
    SMAA on Reshade now comes with these settings by default already (or very close to them, probably a bit better), so actually you don't need to tweak anything. I had forgotten that.
     
  20. MrBonk

    MrBonk Guest

    Messages:
    3,385
    Likes Received:
    283
    GPU:
    Gigabyte 3080 Ti
    A couple updates.


    Return to Castle Wolfenstein -
    MSAA,HSAA
    In order for these to work you need to set
    Code:
    OGL_OCGCONTROL_DIRECT_NVIR
    to
    Code:
    Force Enable
    In my screens I am using this texture pack

    This seems to be the new driver default, but it's there anyway.
    If you try to use SGSSAA with this set, you get of course the infamous texture blurring bug.
    You used to be able to fix it by setting NVIR to disable.
    But now it produces visual garbage like this *only* when trying to force AA. Without forcing AA the image looks totally fine.
    https://u.cubeupload.com/MrBonk/DIRECTNVIRDISABLEWIT.png
    There is a hidden driver flag for changing OGL colorthresholds for unorm8 surfaces (Which is similar at least in name to the function we call the "AA fix"). And that can affect how this looks when set to the 2nd of 2 available settings 0x000000FF
    https://u.cubeupload.com/MrBonk/ioWolfSPx64202204251.png

    According to DRST, the NVIR Enable/Disable bit does:
    It seems to relate how the driver converts shader instructions into what I assume is assembly for the GPU to run.For what it's worth, there are actually functions in the driver for this setting for DX9 and 10/11 as well. I wonder if setting that might affect forced AA in any way as well.

    After mucking around with this, I feel the need to try and get in touch with someone at Nvidia's Driver team to see if anyone can fix the bug introduced with whatever driver it was that broke the old NVinst code.
    I might put my 2080 back in the system and nudge my way forward from 411.63 to figure out which driver broke it.
    I also discovered the hidden
    Code:
    0x0000002C AA_MODE_METHOD_MIXEDSAMPLE_64X
    also works in this game and looks rather good. It will probably work in other OpenGL 1.2 games as well.

    No AA
    8xMSAA
    8xSGSSAA
    8xSGSSAA+ positive 3.00 LOD bias (N64 edition)
    32xS
    64xS


    ________________

    Lost Winds
    0x00000043 - MSAA/TrSSAA
    0x000002C1 - SGSSAA/OGSSAA

    Fairly straightforward here. The 2 for SGSSAA actually improves AA quality a bit. (I think this is the bit for 16RGBAF surfaces IIRC)

    No AA
    8xMSAA
    C1 8xSG
    2C1 8xSG
    4x4 OGSSAA
     

Share This Page