NVidia Anti-Aliasing Guide (updated)

Discussion in 'Videocards - NVIDIA GeForce Drivers Section' started by Cyberdyne, Jan 29, 2012.

  1. Darren Hodgson

    Darren Hodgson Ancient Guru

    Messages:
    16,007
    Likes Received:
    526
    GPU:
    EVGA GTX 1080 Ti SC2
    @Cyberdyne - Thanks for the updated thread.

    You might want to add an extra step in that illustration in post #1 though to remind people that the "Antialiasing - Behaviour Flag" needs to be set to "None" for the Override flag to work as NVIDIA has intentionally set this flag so that you can't force AA for some games (e.g. "Treat .... as ....").
     
  2. Cyberdyne

    Cyberdyne Ancient Guru

    Messages:
    3,582
    Likes Received:
    298
    GPU:
    2080 Ti FTW3 Ultra
    Alright. No problem.
     
  3. WhiteLightning

    WhiteLightning Don Illuminati Staff Member

    Messages:
    28,802
    Likes Received:
    1,651
    GPU:
    GTX1070 iChillx4
    Does anyone know how to force AA wit Need 4 speed the run ?
     
  4. Phreakuency

    Phreakuency Master Guru

    Messages:
    668
    Likes Received:
    0
    GPU:
    Asus Titan
    Is the 'force AA' mode in the Nvidia Control Panel supposed to work in every game?

    Or is that why NVInspector is used, do the flags force AA properly?

    And a different question, even when AA is on in games, am I right in saying some jaggies just can't be fixed?

    Mafia 2 for example, looks great up close but in the distance no setting seems to fix the jaggy/flickery phone lines etc

    Star Wars TOR, Tattooine's sand dunes are jaggy and move like crazy.
     

  5. Darren Hodgson

    Darren Hodgson Ancient Guru

    Messages:
    16,007
    Likes Received:
    526
    GPU:
    EVGA GTX 1080 Ti SC2
    It's a DX11 engine so forcing standard AA is not easy (it may be possible but I'm not aware of a flag as yet). However, the driver's built in FXAA does work so just open NVIDIA Inspector, go to the game profile and set 'NVIDIA Predefined FXAA Usage' to 'FXAA_ALLOW_ALLOWED' and 'Toggle FXAA on or off' to 'FXAA_ENABLE_ON' under the Other section at the bottom of the settings.

    This worked fine for the v290.53 betas but I'm not sure it is working properly with the new v295.51 betas. Can't hurt to try it and if it isn't working then you can always use an InjectFXAA or InjectSMAA mod.
     
  6. WhiteLightning

    WhiteLightning Don Illuminati Staff Member

    Messages:
    28,802
    Likes Received:
    1,651
    GPU:
    GTX1070 iChillx4
    Thanks, though i do prefer 'Real AA' over fxaa or smaa. reall aa is much sharper imo without any blurring. well if i cant , i wont have any other choice though lol.
    i thought the game ran in DX9 ? ill have to check fraps tonight on my g15 since im not sure about this.
     
  7. Redemption80

    Redemption80 Ancient Guru

    Messages:
    18,460
    Likes Received:
    253
    GPU:
    GALAX 970/ASUS 970
    No, its Frostbite 2.0, so has no support for DX9/XP.

    FXAA was the best solution i found, never tried SMAA though, but i hear it does blur things less, no difference to me anyway as i prefer a softer image anyway.
     
  8. Darren Hodgson

    Darren Hodgson Ancient Guru

    Messages:
    16,007
    Likes Received:
    526
    GPU:
    EVGA GTX 1080 Ti SC2
    No. It depends on the engine used and/or whether it has an AA compatibility flag set for it in its game profile (DX9 and DX10+ games need their own flags). It tends to be easier to force MSAA/CSAA in older games as they don't use deferred rendering (see below).

    Yes. The standard NVIDIA control panel allows you to force AA but it doesn't allow you to edit the AA flags in the game profiles so you need an editing tool which is where NVIDIA Inspector comes in handy (previously, we used nHancer for doing that but it is no longer supported or updated).

    Yes, normal MSAA and CSAA does a fairly effective job of removing most jagged edges but doesn't work with transparent/alpha textures such as fences or leaves on trees, etc. For that you have to also enable either transparency multisampled AA or, superior but at the cost of performance, transparency supersampled AA. Together they pretty much eliminate almost all jaggies.

    Unfortunately, the switch to deferred rendering engines, which offer improved lighting for a lower performance hit, means that MSAA and CSAA are not easily supported in many games, even if they use DX10/11 (it can be done but it is very demanding on memory and performance). This is why you'll see more and more games use FXAA or MLAA instead. This type of AA is less effective than MSAA or CSAA because it is a post-process effect that it applied to the entire screen after it has been rendered rather than to each polygon as it being rendered as before. This can result in a slight blurring of the image as it isn't just targeting edges but everything on the screen, including textures and text. As such it can be a bit hit and miss at times but is good if it can be used in conjunction with MSAA/CSAA, like in Battlefield 3 which runs under DX11 for example, as it has a far lower overhead than either transparency MSAA or SSAA.
     
    Last edited: Feb 2, 2012
  9. wasteomind

    wasteomind Master Guru

    Messages:
    384
    Likes Received:
    12
    GPU:
    EVGA RTX 2080 ti
    Does anyone know if Kingdoms of Amalur: Reckoning needs an AA flag? In the demo I've been forcing AA without a flag and it seems to do a decent job.

    However I've had the AA all the way up to 12xs (2x2ss with 4x OGMSAA) and still see jaggies on certain objects like bow strings and some edges on bridges and such. Overall it looks smooth, but the last few area with jaggies tend to stand out more because of it.

    The game runs smooth though, even with 12xS @ 1920x1080 I get 60fps solid. Stepping up to 3x3 SS drops off 15fps and runs pretty solidly between 40-50fps with no real noticeable benefit to IQ.
     
  10. Phreakuency

    Phreakuency Master Guru

    Messages:
    668
    Likes Received:
    0
    GPU:
    Asus Titan
    Thanks Darren, really good post and makes sense, much appreciated!

    Now am I right in saying the actual flags just activate AA in each game and we can still decide what AA to use in the settings below the flag?
     

  11. MrBonk

    MrBonk Ancient Guru

    Messages:
    3,189
    Likes Received:
    169
    GPU:
    MSI RTX 2080
    I am having a strange issue now with Unreal Engine 3 games.
    Specifically UT3.
    Using forced 4x AA, things will look perfectly AA'd but once in Motion AA(or at another angle) looks like it's off.
    (look at the edges of the green object)
    http://img828.imageshack.us/img828/4223/ut3demo2012013121352539.png
    http://img339.imageshack.us/img339/1949/ut3demo2012013121352817.png
    http://img88.imageshack.us/img88/9591/ut3demo2012013121353379.png
    http://img837.imageshack.us/img837/4469/ut3demo2012013121353976.png
    I recall someone having a similar issue with Mass Effect...

    What the hell?


    Driver 295.51
    Intel Core i7 950 @4.2Ghz
    ASUS Rampage Gene II
    6GB G-skill RAM @ 9-9-9-24-1N @1600MHz
    EVGA FPB GTX 460 1GB V1



    ALSO: My brother's PC which has 290.53 setup on it: On Star Wars Battlefront II, I have it set for forced 4XMSAA+4XSGSSAA and everything is completely smoothed but distant objects which are still aliased. When you get close to them they are not aliased..
     
  12. Darren Hodgson

    Darren Hodgson Ancient Guru

    Messages:
    16,007
    Likes Received:
    526
    GPU:
    EVGA GTX 1080 Ti SC2
    Yes but there are different kinds of AA and depending on the flag used their quality can vary. This is why if you look down the list in post #1 some games have multiple AA flags. Some flags only work with specific types of AA or on 16:9 and not 16:10 screen resolutions.

    The ability to edit the AA flags and experiment is, IMO, what puts NVIDIA's drivers way ahead of AMD's. I can pretty much use AA in virtually every one of my games and those that I can't, such as GTA IV, I use FXAA instead.
     
  13. Darren Hodgson

    Darren Hodgson Ancient Guru

    Messages:
    16,007
    Likes Received:
    526
    GPU:
    EVGA GTX 1080 Ti SC2
    @MrBonk - Anti-aliasing in Unreal Engine 3 games tend to be a bit hit and miss in my experience due to the use of deferred rendering which doesn't play nice with it. It's been a while since I've loaded up Unreal Tournament 3 but I remember forced 8xMSAA + TrMSAA working quite well in that game though, there was always the odd edge that looked like it didn't have any anti-aliasing at all. It is the same in other UE3 games too such as Mass Effect 2, X-Men Origins: Wolverine and Batman: Arkham Asylum/City.

    You may also need to use transparency multi- or super-sampling if you're not using it already.

    Or are you saying that it was fine with the 290.53 or earlier drivers but now doesn't look as good with the 295.51 betas?
     
  14. MrBonk

    MrBonk Ancient Guru

    Messages:
    3,189
    Likes Received:
    169
    GPU:
    MSI RTX 2080
    Well I am forcing 4xTrSSAA AFIK. And I can't remember now if I had this before on 275.33 (which was what I upgraded from.)

    I guess I could roll back sometime soon and check. Or download the UT3 Demo on my brother's PC that has a GTS450 and check with forced AA+Transparency if it has the same issue.

    EDIT: I discovered by using the Bioshock Flag, that using SGSSAA fixes the issue but with the that flag performance is unacceptable. SO, I tried TrSSAA and that gave me near the same issue as before with aliasing.
    After that I gave the original UT3 flag a try with SGSSAA and did fix the aliasing issue with great performance.



    I guess the issue is that there is too much shader/specular/etc aliasing than regular trSSAA can handle
     
    Last edited: Feb 3, 2012
  15. flimbo

    flimbo Master Guru

    Messages:
    759
    Likes Received:
    7
    GPU:
    MSI GTX 970
    Just noticed a problem with LA Noire today.
    I updated to the latest 295.51 drivers and in the process lost all my Nvidia profile settings so am redoing them.
    I had LA Noire running nicely with some AA flag but after trying the ones recommended here, I am getting glitchy shadow bugs when using sparse grid supersampling.
    On the vice narcotics case "The Set Up" on the intro, the boxers forearms are bright pink and have weird shadows on them.
    Any ideas on what flag and AA settings to use?
     
    Last edited: Feb 8, 2012

  16. Anarion

    Anarion Ancient Guru

    Messages:
    13,602
    Likes Received:
    376
    GPU:
    GeForce RTX 3060 Ti
    I have the same issue in UT3 and Mass Effect 1. There's a some kind of bug that causes AA not to work at all in I enable v-sync. It flickers on and off without it and if I limit frame rate to 30 FPS it will stay on. Doesn't make any sense.

    It is a fact that UE3 AA will not look as good as in non deferred shading games but that is not the case this time.

    I personally use 8xMSAA+4xTrSSAA in those games but actually it has nothing to do with TrSSAA. AA will still not work properly.
     
    Last edited: Feb 5, 2012
  17. MrBonk

    MrBonk Ancient Guru

    Messages:
    3,189
    Likes Received:
    169
    GPU:
    MSI RTX 2080
    that sucks.

    I created a custom resolution on my monitor of 2560x1440 so I can use downsampling in combination with aa (Sometimes FXAA or SMAA too)and stuff to help a little bit.

    Which is an ok semi-solution when my monitor's native resolution is 1600x900. 2.5 times the amount of pixels.
     
  18. sparrow1911

    sparrow1911 Member Guru

    Messages:
    106
    Likes Received:
    0
    GPU:
    ZOTAC GTX 750 Ti 2GB
    Kingdoms of Amalur AA flag?
     
  19. wasteomind

    wasteomind Master Guru

    Messages:
    384
    Likes Received:
    12
    GPU:
    EVGA RTX 2080 ti
    I posted earlier in this thread about that game and got no response:

    EDIT Nevermind, forget what I said. Forced AA doesn't seem to be working at all. The performance hit is there, the blurring for SSAA and SGSSAA are there, but it doesn't seem to actually be doing anything to clean up edges. FXAA nor SMAA seem to work either. FXAA instacrashes and SMAA creates a huge gamma/brightness spike and doesn't seem to clean up any edges. I've tried a few flags for MSAA/SGSSAA/SSAA and haven't had any luck.
     
    Last edited: Feb 6, 2012
  20. gbooster

    gbooster Member Guru

    Messages:
    115
    Likes Received:
    0
    GPU:
    zotac 1080gtx
    Try the stalker profile 0x0000F0C1. I have it working with 4xMS and Transparency MS. There are still a few seams with jaggies but it works pretty well.
     

Share This Page