nVidia's FXAA mode (anti-aliasing)

Discussion in 'Videocards - NVIDIA GeForce Drivers Section' started by TheHunter, Jul 20, 2011.

  1. CeeJay.dk

    CeeJay.dk Master Guru

    Messages:
    689
    Likes Received:
    8
    GPU:
    Radeon 6870
    I just got SMAA + LumaSharpen + the effects from FXAATool partly working, and instead of letting you guys wait I decided to post an early beta.

    I call this new übershader ShaderFX

    This is version 0.2 beta.
    SMAA, Technicolor, Tonemap, Sepia and Vignette works right.
    LumaSharpen, Bloom and HDR still needs more work.

    The last three shaders sample multiple pixels, but from the original image before SMAA was applied and I need to correct for that or the end result will not look right.

    You can still try and enable them (they compile and run) but the results from them are not correct.. yet.

    I hope to have at least Lumasharpen working again before the week is over and then I start on porting ShaderFX to DirectX10/11
     
  2. Bat_Zonko

    Bat_Zonko Active Member

    Messages:
    87
    Likes Received:
    0
    GPU:
    Sapphire 7950 OC
    This is awesome news. I love the Luma Sharpen you put together (I use it in the FXAA Injector and is soooo nice).
    Tying it into the SMAA injectory (and adding in the tonemapping and other post-effects) is great news.

    Thanks for the hard work. I can't wait to try it out.

    Bats
     
  3. draegunstrife

    draegunstrife Active Member

    Messages:
    66
    Likes Received:
    0
    GPU:
    EVGA 780 Classified
    Thanks CeeJay! Keep up the hard work. Take your time, and get some rest inbetween =D.

    Anyway, can't wait till all are working fine together.
     
  4. TFL Replica

    TFL Replica Master Guru

    Messages:
    385
    Likes Received:
    2
    GPU:
    RTX 3060 Ti
    Sweet. Thank you for keeping us updated!
     

  5. CeeJay.dk

    CeeJay.dk Master Guru

    Messages:
    689
    Likes Received:
    8
    GPU:
    Radeon 6870
    If you have succesfully combined FXAA and Lumasharpen so you can use both at the same time, then I would love to see the code - it might help me figure out why my own FXAA + Lumasharpen attempt doesn't yet work right.

    I probably shouldn't have taken the shaders apart and put them back again in my own code framework, but simply stuck with the original code for that - I'm guessing I forgot to dot an i somewhere.

    I postponed working on combining FXAA and Lumasharpen once I realised I was very close to getting SMAA working right, because of the two I consider SMAA to be the superior AA shader, but I know that not everyone agrees.
    Once I get everything to work right with SMAA and get it ported to DX10/11 then I'll give some attention to FXAA again.
     
  6. CeeJay.dk

    CeeJay.dk Master Guru

    Messages:
    689
    Likes Received:
    8
    GPU:
    Radeon 6870
    BTW I could use some help from you guys.

    I want ShaderFX to include some good default settings and some good presets (this is an upcoming feature)

    So if you have tweaked the settings and found some settings you like, please share them with each other and myself so I can study them and make some good presets from them in the future.

    I of course also need feedback if something is broken and doesn't work right, and I also welcome suggestion to new and better shader effects and features to include in the future. I have a long wishlish myself that I start working on after ShaderFX is out of beta.
     
  7. CeeJay.dk

    CeeJay.dk Master Guru

    Messages:
    689
    Likes Received:
    8
    GPU:
    Radeon 6870
    Looked up ShaderFX today at work and there are already other things named ShaderFX

    My shader mod is going to need a new name.
    Any suggestions?
     
  8. yosef019

    yosef019 Ancient Guru

    Messages:
    2,156
    Likes Received:
    10
    GPU:
    EVGA 1080 ti
    AAShader

    can you make smaa cmfortable with msi afterburner?
     
  9. CeeJay.dk

    CeeJay.dk Master Guru

    Messages:
    689
    Likes Received:
    8
    GPU:
    Radeon 6870
    To make MSI Afterburner work with my mod and other mods that use a modified Direct3D dll (FXAATool, InjectFXAA, InjectSMAA, ENB) you need to :
    1) Update to the latest version of MSI Afterburner.
    2) Start MSI Afterburner.
    3) Switch to the "MSI On-Screen Display Server" window.
    4) Create a new profile for your game and change to it.
    5) Click the big wrench icon to change advanced settings.
    6) Go to General -> Compatibility properties and turn on "Enable compatibility with modified Direct3D runtime libraries".

    MSI Afterburner is now compatible with shader mods.

    You can also change the setting for the Global profile, but MSI does not recommend this because it might prevent some Direct3D applications from starting.
     
  10. Bat_Zonko

    Bat_Zonko Active Member

    Messages:
    87
    Likes Received:
    0
    GPU:
    Sapphire 7950 OC
    Oh boy. Well, it is a complete hack job, but here goes.

    From your LumaSharpen shader.fx file, I took the #define and sampler lines and placed them in the FXAA shader.fx file.

    I followed the setup of the FXAA shader.fx so that the #define lines are follwing the initial #include statements for the settings.

    The sampler calls (?) from the Lumasharpen I placed in the same sampler call area of the original FXAA shader.fx file.

    Then, in the post.h file that calls the additional shaders, I replaced the original pre-sharpen with the your call to include the Lumasharpen.h in your shader.fx file.

    Your Lumasharpen.h file I placed in the injFX_Shaders sub-folder.

    It is a complete "I don't know sh*t" hack job of trying to follow the variables and returned/passed values. It seems to work fine, and IMHO looks great (really good in Fallour NV).

    Here are the hacked-up shader.fx file and the post.h file. I use the FXAA Injector that Mineria released on the Google code site.

    Shader.fx
    http://dl.dropbox.com/u/43021579/shader.fx
    Post.h
    http://dl.dropbox.com/u/43021579/Post.h

    I probably did it all wrong and it isn't working right. But, it looks good :)

    Bats
     

  11. Cyrosis

    Cyrosis Ancient Guru

    Messages:
    1,560
    Likes Received:
    1
    GPU:
    HD4000
    Love your Luma Sharpen injector, CeeJay.dk. Recently tried using it to alleviate the blur from super sampling, and it works spectacularly; I was flipping two screenshots back and forth to compare MSSA vs SSAA + your injector, and there was essentially no loss in texture quality and no sharpening artifacts. Other sharpening injectors were full of contrast artifacts, made text ugly, and just generally didn't bring the detail back like Luma Sharpen does.
     
  12. CeeJay.dk

    CeeJay.dk Master Guru

    Messages:
    689
    Likes Received:
    8
    GPU:
    Radeon 6870
    New version of InjectSMAAmod - now version 0.2
    https://dl.dropbox.com/u/9942986/injectSMAAmod.7z

    The new version allows for and enables by default color edge detection.

    This improves detection of edges that the normal luma edge detection would sometimes miss, but it runs slightly slower.

    I felt the higher quality was worth it and so I have enabled it by default, but you can switch it on and off as you like.
     
    Last edited: Aug 3, 2012
  13. BetA

    BetA Ancient Guru

    Messages:
    4,376
    Likes Received:
    312
    GPU:
    G1-GTX980@1400Mhz
    comparsion screens?
    can u show off some stuff?

    im really interested in that version..also how do i set it up to have the max avaiable quality?

    greetz BetA
     
  14. TFL Replica

    TFL Replica Master Guru

    Messages:
    385
    Likes Received:
    2
    GPU:
    RTX 3060 Ti
    Try it out yourself. Open the files in notepad++ and you'll see. One of them allows the same configuration would have with the regular injectSMAA and another one allows fine tuning (if the preset is set to custom).
     
  15. BetA

    BetA Ancient Guru

    Messages:
    4,376
    Likes Received:
    312
    GPU:
    G1-GTX980@1400Mhz
    in the injector.ini i can set to ultra, yes i know that..but i want to keep the custom cause i think this color or luma detection thing is better..still i cant figure out wich file i have to edit..its toooooooooooo much information in these files and i have no clue at all where to start...

    anyway, gonna test it on prototype 2 in a bit...

    EDIT:

    i made some comparsions now..well..i dont see any diff compared to smaa @ ultra at all...on some lines, normal smaa looks better i think..not shure..here are the screens...

    INGAME AA HIGH:
    [​IMG]

    SMAA-ULTRA:
    [​IMG]

    SMAA-Custom-Mod (or what u call it):
    [​IMG]


    -------------------------------------------------------------------
    -------------------------------------------------------------------

    Zoom´s...

    INGAME AA HIGH:

    [​IMG]


    SMAA ULTRA:

    [​IMG]


    SMAA CUSTOM MOD:

    [​IMG]



    WHat u think?
     
    Last edited: Aug 3, 2012

  16. Anarion

    Anarion Ancient Guru

    Messages:
    13,605
    Likes Received:
    382
    GPU:
    GeForce RTX 3060 Ti
    Hard to say... Use pixel resize/nearest neighbour resampling (double or quadruple the original resolution) and post png's. Too many compression artifacts there.
     
    Last edited: Aug 3, 2012
  17. yosef019

    yosef019 Ancient Guru

    Messages:
    2,156
    Likes Received:
    10
    GPU:
    EVGA 1080 ti
    Thanks
     
  18. yosef019

    yosef019 Ancient Guru

    Messages:
    2,156
    Likes Received:
    10
    GPU:
    EVGA 1080 ti
    Same,
    i finish the game so now i less care about aa
    ingame aa + smaa shood do the work ?
     
  19. CeeJay.dk

    CeeJay.dk Master Guru

    Messages:
    689
    Likes Received:
    8
    GPU:
    Radeon 6870
    The SMAA presets define these values :

    Code:
    LOW:
    #define SMAA_THRESHOLD 0.15
    #define SMAA_MAX_SEARCH_STEPS 4
    #define SMAA_MAX_SEARCH_STEPS_DIAG 0
    #define SMAA_CORNER_ROUNDING 100
    
    MEDIUM:
    #define SMAA_THRESHOLD 0.1
    #define SMAA_MAX_SEARCH_STEPS 8
    #define SMAA_MAX_SEARCH_STEPS_DIAG 0
    #define SMAA_CORNER_ROUNDING 100
    
    HIGH:
    #define SMAA_THRESHOLD 0.1
    #define SMAA_MAX_SEARCH_STEPS 16
    #define SMAA_MAX_SEARCH_STEPS_DIAG 8
    #define SMAA_CORNER_ROUNDING 25
    
    ULTRA:
    #define SMAA_THRESHOLD 0.05
    #define SMAA_MAX_SEARCH_STEPS 32
    #define SMAA_MAX_SEARCH_STEPS_DIAG 16
    #define SMAA_CORNER_ROUNDING 25
    You can use them as guidelines for finding your own preset.

    Some hints from the SMAA.h file :

    Code:
    /**
     * SMAA_THRESHOLD specifies the threshold or sensitivity to edges.
     * Lowering this value you will be able to detect more edges at the expense of
     * performance. 
     *
     * Range: [0, 0.5]
     *   0.1 is a reasonable value, and allows to catch most visible edges.
     *   0.05 is a rather overkill value, that allows to catch 'em all.
     *
     *   If temporal supersampling is used, 0.2 could be a reasonable value, as low
     *   contrast edges are properly filtered by just 2x.
     */
    #ifndef SMAA_THRESHOLD
    #define SMAA_THRESHOLD 0.1
    #endif
    
    /**
     * SMAA_DEPTH_THRESHOLD specifies the threshold for depth edge detection.
     * 
     * Range: depends on the depth range of the scene.
     */
    #ifndef SMAA_DEPTH_THRESHOLD
    #define SMAA_DEPTH_THRESHOLD (0.1 * SMAA_THRESHOLD)
    #endif
    
    /**
     * SMAA_MAX_SEARCH_STEPS specifies the maximum steps performed in the
     * horizontal/vertical pattern searches, at each side of the pixel.
     *
     * In number of pixels, it's actually the double. So the maximum line length
     * perfectly handled by, for example 16, is 64 (by perfectly, we meant that
     * longer lines won't look as good, but still antialiased).
     *
     * Range: [0, 98]
     */
    #ifndef SMAA_MAX_SEARCH_STEPS
    #define SMAA_MAX_SEARCH_STEPS 16
    #endif
    
    /**
     * SMAA_MAX_SEARCH_STEPS_DIAG specifies the maximum steps performed in the
     * diagonal pattern searches, at each side of the pixel. In this case we jump
     * one pixel at time, instead of two.
     *
     * Range: [0, 20]; set it to 0 to disable diagonal processing.
     *
     * On high-end machines it is cheap (between a 0.8x and 0.9x slower for 16 
     * steps), but it can have a significant impact on older machines.
     */
    #ifndef SMAA_MAX_SEARCH_STEPS_DIAG
    #define SMAA_MAX_SEARCH_STEPS_DIAG 8
    #endif
    
    /**
     * SMAA_CORNER_ROUNDING specifies how much sharp corners will be rounded.
     *
     * Range: [0, 100]; set it to 100 to disable corner detection.
     */
    #ifndef SMAA_CORNER_ROUNDING
    #define SMAA_CORNER_ROUNDING 25
    #endif
    For SMAA_MAX_SEARCH_STEPS , and SMAA_MAX_SEARCH_STEPS_DIAG the rule is that higher numbers = higher quality, but there are diminishing returns and very high number don't make that big a difference over more reasonable numbers, but very low numbers hurt quality noticeably.
    I suggest staying above 2 for both values as any lower hurts quality a lot.
    As for high high you want to go, I don't know. I find it hard to spot the difference between 6 and 8 and very hard to see the difference between 8 and 16.

    SMAA_CORNER_ROUNDING is a personal preference. If you like more round corners then choose a high number (max is 100) and if you like sharp corners choose a low value (minimum is 0)
    This mostly affects text and other small UI details. I prefer a very low value.

    SMAA_THRESHOLD is probably the most important value. Low values find more edges to smooth, but too low values may find false edges that should not be smoothed, like details in textures.
    Higher values find less edges and preserve more of the original image intact and make the shader run a little faster because it has less edges to smooth, but too high values don't find aliased edges that should have been smoothed.
    The best threshold will probably vary from game to game.
    The information in SMAA.h is correct .. a value of 0.05 is a bit extreme and find lots of false edges that should not have been smoothed.
    I think that you should probably stay between 0.08 and 0.20.
    My guess is that around 0.12 to 0.15 is a good blend between smoothing enough aliased lines and maintaining fine details in the image.

    COLOR_EDGE_DETECTION 0 makes the shader use Luma edge detection where it decides if a pixel is part of an edge based on the difference in luma (it's intensity/brightness) between it and the neighboring pixels.
    COLOR_EDGE_DETECTION 1 instead determines the same thing but based on the difference in color values. This is slightly slower but may catch some edges that luma edge detection will not.

    SMAA_DIRECTX9_LINEAR_BLEND 1 is necessary if you use DX9 hardware.
    DX9

    From SMAA.h :
    Code:
    /**
     * In the last pass we leverage bilinear filtering to avoid some lerps.
     * However, bilinear filtering is done in gamma space in DX9, under DX9
     * hardware (but not in DX9 code running on DX10 hardware), which gives
     * inaccurate results.
     *
     * So, if you are in DX9, under DX9 hardware, and do you want accurate linear
     * blending, you must set this flag to 1.
     *
     * It's ignored when using SMAA_HLSL_4, and of course, only has sense when
     * using sRGB read and writes on the last pass.
     */
    Most people today have DX10 or DX11 hardware and should keep this value at 0, even if they run DX9 games.

    People who use Nvidia Geforce 7-series and older and people who use ATI Radeon X1xxx-series and older should enable this.

    If you have a Geforce 8-series or a ATI/AMD Radeon HD2000-series or newer, keep this disabled.
     
    Last edited: Aug 17, 2012
  20. CeeJay.dk

    CeeJay.dk Master Guru

    Messages:
    689
    Likes Received:
    8
    GPU:
    Radeon 6870
    I noticed that are some subpixel details like the fences that disappear because they are thinner than a pixel. FXAA and SMAA can't help you with those.

    Try combining MSAA or SSAA with SMAA.
    Or better yet, try downsampling together with SMAA - There is a guide here elsewhere on the forum.
     

Share This Page