NVidia Anti-Aliasing Guide (updated)

Discussion in 'Videocards - NVIDIA GeForce Drivers Section' started by Cyberdyne, Jan 29, 2012.

  1. MrBonk

    MrBonk Ancient Guru

    Messages:
    3,191
    Likes Received:
    169
    GPU:
    MSI RTX 2080
    The 8 usually has to be present to work in UE3.
    It's also possible that forced AA is conflicting with post processing in the game and you can't always work around that. It may not always get all aliasing as well. (Such as aliasing from Post Processing or shaders)
    (In addition, if it's one of the UE3 games that has that stupid ubershader nonsense where you can't disable individual PP effects, you may or may not be getting forced FXAA as well, which definitely will affect how the forced AA looks. As opposed to laying FXAA on top as you would with Reshade or the driver when using AA. You'd have to disable all PP to disable FXAA).
    It looks decent from the comparisons in the reference post
    https://abload.de/img/thebureau0x000000003us90.png
    https://abload.de/img/thebureau0x080010c1lzsc7.png
    It's clearly getting a lot of sources of aliasing, though there still are some rough edges and some aliasing on the bloom to the right.
     
  2. Tizoc

    Tizoc Active Member

    Messages:
    77
    Likes Received:
    7
    GPU:
    GTX970M
    Oh yeah your first image is kinda what I hoped to get. If you don't mind me asking, what did you set up nvidia inspector to and was there something you disabled in game to achieve that please?
     
  3. MrBonk

    MrBonk Ancient Guru

    Messages:
    3,191
    Likes Received:
    169
    GPU:
    MSI RTX 2080
    I am not the author of those images, they were taken from the 3DCenter forum post in the reference in the document.
    Do you mind posting an exact A/B comparison of what you are seeing? (Make sure to use as close to a 100% repeatable scenario as possible for proper comparison vs no AA vs off)
     
  4. spiritmoon2209

    spiritmoon2209 New Member

    Messages:
    5
    Likes Received:
    1
    GPU:
    GTX 980 Ti
    A bit late to report, but I used the flag 0x080100C1 for Ys SEVEN (I am not sure about this, but I used SpecialK together with ReShade (latest) to have 10-bit format output).

    WIth the flag, I could use SGSSAAx8 (I used 8xQ, so Quincunx MSAA) and it does help remove a lot of shimmering edges in the game.

    I would also like to add that we can enhance Trails in the Sky the 3rd by editing the .ini setting file FullAntiAlias=4 (always 0 and cannot be changed by the game settings) into OGSSAA/SGSSAA.
     
    Cave Waverider likes this.

  5. MrBonk

    MrBonk Ancient Guru

    Messages:
    3,191
    Likes Received:
    169
    GPU:
    MSI RTX 2080
    Do you have comparisons with Ys Seven? I could never get SpecialK to work with the game, with newer versions or with the test versions that were supposed to fix the lower quality back buffers.

    With The3rd you can probably just force MSAA/SGSSAA like FC/SC in DX8 mode without a flag.
     
  6. spiritmoon2209

    spiritmoon2209 New Member

    Messages:
    5
    Likes Received:
    1
    GPU:
    GTX 980 Ti
    I will try to produce comparisons with lower res, since my native res is 4K so the aliasing is not too bad. Only in motion does SGSSAA have a significant impact to reduce the shimmering, and the GPU utilization is noticeably higher so it should be working.

    I tested several SpecialK versions, the working one is 0.8.66, installed by simply putting d3d9.dll into the game folder. I had to use SpecialK to extract textures and do AI upscaling.
     
    Last edited: Dec 16, 2020
  7. MrBonk

    MrBonk Ancient Guru

    Messages:
    3,191
    Likes Received:
    169
    GPU:
    MSI RTX 2080
    You could still see a difference in a still shot even at 4k if it's easier to produce a comparison. (Just compress the jpg at 90-100% to save a little space).

    I was able to get it to inject and it's upgrading the depth buffers to 24 bit clearly, only 8 bit color. But forcing AA definitely does not work.
     
    Last edited: Dec 17, 2020
  8. spiritmoon2209

    spiritmoon2209 New Member

    Messages:
    5
    Likes Received:
    1
    GPU:
    GTX 980 Ti
    Here's the comparison. By zooming in I can see the difference at the edges of the mountain and the blades of the windmill.

    https://imgur.com/a/4gKKZwq

    (The first image showed that we can force even a 10-bit color framebuffer using reshade. (I installed reshade with d3d9.dll and specialK with dinput8.dll and then add

    [APP]
    Force10BitFormat=1

    to d3d9.ini. This helps with banding for many games if the monitor can display 10-bit colors).

    You can also see the shimmering in action here: (unfortunately I can only capture in 1080p60 using Xbox game bar, NVIDIA shadowplay is not playing nice with SpecialK, and I haven't tested injection without SpecialK / ReShade yet :D)

    (with SGSSAA)

    (default).
     
    Last edited: Dec 17, 2020
  9. MrBonk

    MrBonk Ancient Guru

    Messages:
    3,191
    Likes Received:
    169
    GPU:
    MSI RTX 2080
    Yeah that definitely looks like it is working.
    I tried with Reshade and got them to play along. But none of my monitors I can connect to my PC can accept a 10-bit signal. Only 12-bit (Which is dithered probably), so it just crashes on launch when I try to force the 10-bit mode. Too bad it can't just dither it down instead. I'd really love to get AA working in this game.
     
  10. spiritmoon2209

    spiritmoon2209 New Member

    Messages:
    5
    Likes Received:
    1
    GPU:
    GTX 980 Ti
    I just tested again without ReShade and AA still works. Oh and my current TV is also accepting 12-bit signals, so probably there is something different there.

    My Inspector setting is like this. I don't think the AO injection has any effect though.

    So I am not sure if possibly you're using the GOG version which might differ from the Steam version. I think the game was patched sometimes after launch to support 4K, which crashes on the YS build of SpecialK.

    upload_2020-12-18_3-5-28.png
     

  11. MrBonk

    MrBonk Ancient Guru

    Messages:
    3,191
    Likes Received:
    169
    GPU:
    MSI RTX 2080
    using Steam version. But Windows 7. I'm going to try in Windows 10 maybe.
    Yeah no dice. Same stuff, dunno. I'll add it to the document anyway because it clearly seems to work for you. Maybe something special in your SpecialK .inis??
     
    Last edited: Dec 18, 2020
  12. Tizoc

    Tizoc Active Member

    Messages:
    77
    Likes Received:
    7
    GPU:
    GTX970M
    Just checking but what option in Space Marine should I turn off to utilize SSGSS AA in it?
    https://www.forum-3dcenter.org/vbulletin/showpost.php?p=10080069&postcount=5397

    Your screen shots are what I'm getting; some stuff are aliased like bars or furniture and such.
     
  13. MrBonk

    MrBonk Ancient Guru

    Messages:
    3,191
    Likes Received:
    169
    GPU:
    MSI RTX 2080
    You can use it with the PPAA on, but turning off the PP setting should disable the PPAA.

    If you are still seeing Aliasing in X-Com that's when you have to either use 8xSGSSAA or try a hybrid solution involving downsampling. (Like using 4xSGSAAA at 2.25xDSR, or 2xSGSSAA at a higher ratio)
     
  14. Dreadmoth

    Dreadmoth Member

    Messages:
    29
    Likes Received:
    0
    GPU:
    MSI 1070Ti Titanium
    Starpoint Gemini 2 (D3D11, no bits) - SGSSAA works pretty well to fix shader aliasing missed by the built-in MSAA. Also smooths out the half-resolution pixelation in the volumetric fog effect.

    8xMSAA
    4xSGSSAA
    8xSGSSAA

    DSR works, but the game lacks UI scaling and bloom looks strange (radius too small) at high resolutions.
    The game seems to have a frame rate cap at 111 or 112 FPS. Using RTSS to cap at a slightly lower value (used 110 in the 4xSGSAA shot) improves frametime variance.
     
  15. MrBonk

    MrBonk Ancient Guru

    Messages:
    3,191
    Likes Received:
    169
    GPU:
    MSI RTX 2080
    Added, thanks!
     

  16. MrBonk

    MrBonk Ancient Guru

    Messages:
    3,191
    Likes Received:
    169
    GPU:
    MSI RTX 2080
    Learned something interesting through tests today with Shadow Complex Remastered that i'm sure may help out other UE3 games.

    Setting "ShadowTexelsPerPixel=" to a lower value can increase performance and reduce drops due to the shadows when forcing SGSSAA. A value of "0.900000" down from a default of "1.3" on an RTX2080 can drop GPU usage in certain scenes by 20+%. In one particular Tutorial Level (The dashing one) this keeps the GPU from choking when breaking through walls that have broken parts that cast real time shadows. Just walking through that debris casting shadows at 1.3 is enough to bring it down to 37FPS at 2880x1620 with 4xSGSSAA 100% gpu usage. At 0.9 it never drops from 60 and GPU usage is only at around 70%.

    I remember shadows choking with SGSSAA enabled even at lower resolutions on a GTX 980 and I could never figure out why. I think this may be it. Applying this tweak to other UE3 games may increase performance in shadow heavy scenes.

    That opening prologue level though still tanks the GPU however.
    https://www.screenshotcomparison.com/comparison/9786
     
    Last edited: Jan 5, 2021
  17. Tizoc

    Tizoc Active Member

    Messages:
    77
    Likes Received:
    7
    GPU:
    GTX970M
    Ankh Anniversary Edition
    0x0000F0C1

    This AA bit works to smooth out the aliasing in the game, I am running it at 4x SSGSS AA and it looks clean :)
    [​IMG] [​IMG] [​IMG]

    Not much more could be done; it is a PS2-@$$ looking game but at least it has a nice visual design somewhat
     
  18. MrBonk

    MrBonk Ancient Guru

    Messages:
    3,191
    Likes Received:
    169
    GPU:
    MSI RTX 2080
    Cleans up nicely. I'll add it to the list.

    You could try without the F in there though. See if that makes a difference to sharpness.
    Also, try the AA fix as sometimes that will result in a sharper image as well.
     
    Last edited: Jan 7, 2021
    Tizoc likes this.
  19. Tizoc

    Tizoc Active Member

    Messages:
    77
    Likes Received:
    7
    GPU:
    GTX970M
    Lemme test it out with Ankh 2 then.

    So the bit would be
    0x000000C1
    ?
     
    Last edited: Jan 7, 2021
  20. Tizoc

    Tizoc Active Member

    Messages:
    77
    Likes Received:
    7
    GPU:
    GTX970M
    Took a screenshot from early in the game with AA Fix enabled+ the bit mentioned and well...it works to give some needed AA but I'm not noticing much otherwise ^^;

    Still a nice clean presentation~

    [​IMG]
     

Share This Page