Resizeable BAR support issues

Discussion in 'Videocards - AMD Radeon Drivers Section' started by Deleted member 282649, Nov 26, 2020.

Tags:
  1. Edit: The graphics driver version used matters.

    Re-sizable BAR causes broken VRAM reporting and generally breaks games (low performance or start-up crash) with non-DX11 graphics APIs (DX9, DX12, Vulkan) with drivers that don't support it. With a RX 580 (Polaris) and 2700X (Zen+) on Windows 10, Adrenaline driver 20.12.1 works fine, whereas Enterprise driver 20.Q4 doesn't work (my report below is with the unsupported Adrenaline 20.11.1 drivers). On Linux, it seems to work fine on kernels 5.9 and 5.10 (openSUSE TW).

    The quickest way to test if your driver works or not on Windows is to enable Resizable BAR in BIOS/firmware options, open Radeon Control Panel -> Performance -> Metrics. If VRAM is "Infinity", the driver doesn't support Re-sizable BAR. This is only applicable from the Radeon Control Panel (Windows Task Manager and GPU-Z will report a VRAM number regardless of the driver not supporting the option).

    [​IMG]

    I can confirm Resizable BAR working fine with my 2700X, on an ASUS PRIME X470-PRO motherboard with 5833 Beta BIOS (Jan 15, 2021), and RX 580 with 21.1.1 Adrenaline drivers on Windows 10 20H2 as of 2021/01/21. Also looks fine from Linux.

    ______________

    Original post:

    Recently some Ryzen motherboard vendors released beta BIOS files (notably for older chipsets like X470) that adds support for Smart Access Memory (SAM), or specifically Re-sizeable BAR support. I have an ASUS PRIME X470-PRO motherboard that got such a BIOS.

    With a RX 580, I noticed the following issues under Windows:
    1. The VRAM usage reported by Radeon Software's metrics is "Infinity" (screenshot)
    2. A game using DXVK (AoE2DE) flickers a corrupted white screen
    3. Oculus Link doesn't work (flickers white upon starting)
    4. Dota 2 with Vulkan had about 10 FPS (DX11 is about 80-90)
    Specs:
    • ASUS PRIME X470-PRO motherboard (5806 beta BIOS)
    • RX 580 8GB GPU (20.11.1 drivers)
    • Ryzen 7 2700X CPU
    • Windows 10 20H2 (10.0.19042)
     
    Last edited by a moderator: Jan 21, 2021
  2. Fox2232

    Fox2232 Guest

    Messages:
    11,808
    Likes Received:
    3,371
    GPU:
    6900XT+AW@240Hz
    Thanks for report. For now, ASRock did not release BIOS update for X470 chipset based boards, just B450 ones. (Therefore I could not do testing, yet. Unless I pulled out of the box x570 board I have for friend's PC.)

    Your post may help others when they start scratching their heads.

    I have here RX 580 Nitro (non-Plus) which reports itself as 480 when low clock/poser vBIOS is enabled. (Clock matches reference RX 480.)
    So, I would be able to test with similar GPU.
     
  3. JonasBeckman

    JonasBeckman Ancient Guru

    Messages:
    17,564
    Likes Received:
    2,961
    GPU:
    XFX 7900XTX M'310
    Wouldn't you need to enable these features in the bios and possibly whatever is in AMD 20.11.2 itself?

    [​IMG]

    Bios should be something like this, guessing setting BAR to enabled would also set 4G to enabled whereas auto might leave it disabled it 4G is disabled.


    Bios could have changed something though or the defaults could for whatever reason be enabled by default but I was under the impression SAM itself needed a bit to actually activate it.

    Driver wise I'm not sure if there's a toggle or if it just works if the criteria is met (CSM disabled being needed as well?) nor if older GPU's are supported or not.


    EDIT: Yep bios settings and 20.11.2 and other than 4G and BAR the compatibility support module (CSM) has to be disabled and all of that need to be set up for it to work, depending on hardware Re-Size BAR Support might not even show up in the bios.

    Wonder if it's 1.1.8.0 I think perhaps the AGESA code and all it's modules or routines has a few issues that need to be worked out, newer chipset driver might help too for the software side of it after the bios is active such as the PCI Express driver unless the October 19th(?) release already includes this assuming the latest chipset software is installed on the system. :)

    [​IMG]

    About how it looks like from my own Gigabyte bios though the BAR setting isn't there as I don't have a 5000 series Ryzen CPU.
     
    Last edited: Nov 26, 2020
    BlindBison and somemadcaaant like this.
  4. Yep; the settings are in the BIOS, and on my ASUS BIOS, the settings are named the same thing (Above 4G Decoding and Re-Size BAR Support). CSM is disabled in my case also. The above 4G decoding option wasn't present in my BIOS prior to the beta BIOS update.

    Outside of those issues mentioned, I don't know how to check if the Re-Size BAR support or the above 4G decoding options actually do anything beneficial on Windows. On Linux, it seems like something actually happens when adjusting those settings (reported it here on Reddit). I can keep that option enabled and things work ok from Windows, and Linux reports the full 8GB BAR size.

    On Windows, even though Radeon Software's metrics has a broken VRAM counter, Task Manager and GPU-Z seemingly report correct information.

    I'd be curious to see if Vulkan and games works on Linux along with VRAM reporting from AMD's driver with the Resize BAR option enabled, but I'd need to set aside some time for that one day (I think there's 3 different Vulkan drivers for AMD GPUs on Linux iirc). Another test I'm interested in is seeing if anything else breaks on Windows; I didn't test any other game or API.
     

  5. Krteq

    Krteq Maha Guru

    Messages:
    1,130
    Likes Received:
    765
    GPU:
    MSI RTX 3080 WC
    SAM/Resizeable BAR is supported only on RX 6000 series by windows driver. It won't work on RX 580
     
    cucaulay malkin likes this.
  6. Fox2232

    Fox2232 Guest

    Messages:
    11,808
    Likes Received:
    3,371
    GPU:
    6900XT+AW@240Hz
    Well, it apparently gets enabled and changes behavior.
     
    DanielRX likes this.
  7. JonasBeckman

    JonasBeckman Ancient Guru

    Messages:
    17,564
    Likes Received:
    2,961
    GPU:
    XFX 7900XTX M'310
    It does work on Linux at least once these options were available, was trying to look up if there were any known issues but only suggestion was to keep a back up and be careful in case they trim older CPU support.
    https://www.reddit.com/r/Amd/commen...agesa_bios_updates_enables_amd_smart/gdjqwpd/

    From 256 to the full 8192 available with the 8 GB VRAM.

    Which was from here when looking up bios info.
    https://www.reddit.com/r/Amd/comments/k0oyhb/asus_x470_ryzen_5000_agesa_v2_pi_1180_beta_bioses/

    Linux is it's own environment though and Windows is it's thing but it is good to see it working and not just for the newest GPU hardware although I'm not sure if that's the same on Windows though it could be the newer AGESA doesn't restrict anything.
    (But the Windows AMD drivers could still have something.)
     
  8. cucaulay malkin

    cucaulay malkin Ancient Guru

    Messages:
    9,236
    Likes Received:
    5,208
    GPU:
    AD102/Navi21
    how does it change the results ?
     
  9. Fox2232

    Fox2232 Guest

    Messages:
    11,808
    Likes Received:
    3,371
    GPU:
    6900XT+AW@240Hz
    Apparently: bold part in OP shows that it does something. While it comes with those undesirable effects. (At least for now.)
    It means that even older cards than RDNA2 get it enabled. And that's exact opposite from Krteq's statement.

    (But hey, it was apparent from context. All there was needed, was to read OP and very post I quoted together with my post.)
     
  10. cucaulay malkin

    cucaulay malkin Ancient Guru

    Messages:
    9,236
    Likes Received:
    5,208
    GPU:
    AD102/Navi21
    no,the results.
    how much faster is it
     

  11. Fox2232

    Fox2232 Guest

    Messages:
    11,808
    Likes Received:
    3,371
    GPU:
    6900XT+AW@240Hz
    For that, you are asking wrong person.
     
  12. cucaulay malkin

    cucaulay malkin Ancient Guru

    Messages:
    9,236
    Likes Received:
    5,208
    GPU:
    AD102/Navi21
    he should test more games
    it looks like he's enabling something that's not supposed to be enabled.
     
  13. Astyanax

    Astyanax Ancient Guru

    Messages:
    17,011
    Likes Received:
    7,351
    GPU:
    GTX 1080ti
    and a 6800.
     
    cucaulay malkin likes this.
  14. Fox2232

    Fox2232 Guest

    Messages:
    11,808
    Likes Received:
    3,371
    GPU:
    6900XT+AW@240Hz
    AMD stated that it will get enabled automatically moment compatible HW scenario is detected. No need to flip any options in OS/Driver.
     
  15. With the calculations I did here, it seems like:
    • BAR defaults to 256MB
    • Presumably the BAR is limited by-default to that space if the BIOS doesn't advertise above 4G decoding
    • If I vaguely recall correctly, this limit can increased from Linux regardless of BIOS support with some kernel option (I know I messed with the option in the past with a Macbook and eGPU)
    • I recall Windows not increasing that BAR limit on my Macbook, preventing the eGPU from working with it (I needed to do something with DSDT modding to fix it)
    • If 4G decoding is enabled in BIOS, Linux still starts with the 256MB BAR, but releases it, and allocates a larger size later on
    • If 4G decoding and re-sizable BAR is enabled in BIOS, it seems the BAR is set to a large amount BIOS-side, and Linux goes with it (but it seems AMD's driver on Windows doesn't handle it well)

    I'm not too sure what would be affected by a larger BAR, but it sounds like anything having high VRAM usage might benefit? poclmembench on Windows still reports the same numbers seemingly (205-206GByte/s).

    In Device Manager under the Resources tab for the GPU, it has memory ranges; I don't know how these get set, but I have two theories:

    • Those memory ranges are set at device install time, and stay that way until a new driver install (so any BIOS BAR stuff won't apply automatically)
    • The PCI Lock option in msconfig would do something to affect that

    Edit: The memory ranges in Device Manager do change based on the BIOS settings. PCI Lock option doesn't fix the infinity VRAM report with 4G and resize BAR enabled. Not sure how to convert the memory range numbers in Device Manager to something readable yet.
     
    Last edited by a moderator: Nov 26, 2020
    Fox2232 likes this.

  16. Here's the memory ranges (screenshots):
    The memory ranges changed every time. When enabling the re-sizable BAR option, a new "Large Memory Range" label appears under Resources for the first memory range. So this seems to at least be one observation that the resizable BAR option does something notable technically.

    PCI Lock is enabled for all those memory range screenshots (presumably in order to make sure Windows respects the BIOS ranges). Upon enabling this option though, both my motherboard's HDA controller and the HDA controller on the GPU don't start ("Insufficient system resources exist to complete the API."), but seemingly every other device starts fine; this happens regardless of the 4G decoding and resizable BAR options.

    If the memory ranges can be converted to a readable size, I wonder if it would show the BAR sizes.

    Seems there's all sorts of trees to look at from Device Manager pointing at different PCI bridges and resources, but Windows doesn't make debugging easy for this sort of thing.

    On a different note, the kernel options for Linux I recall messing with in the past for a Macbook and eGPU was pci=hpbussize=10,hpmemsize=2M,nocrs,realloc (from this thread); might be useful info for something.
     
    Last edited by a moderator: Nov 26, 2020
  17. Astyanax

    Astyanax Ancient Guru

    Messages:
    17,011
    Likes Received:
    7,351
    GPU:
    GTX 1080ti
    PCI Lock intentionally disables the ability for thte OS to perform PNP IO remapping.
     
  18. JonasBeckman

    JonasBeckman Ancient Guru

    Messages:
    17,564
    Likes Received:
    2,961
    GPU:
    XFX 7900XTX M'310
    Hmm maybe there's more to what AMD does with SAM or perhaps there's additional improvements with Zen3 here.
    https://www.reddit.com/r/Amd/comments/k1zutr/rumor_regarding_support_of_sam_for_older_gen_zen/

    Interesting. :)
    Could still work on older CPU's probably will once AMD adds more support and how it all works out and what GPU's it gets enabled for in the display driver part of this little chain of requirements but that is a significant difference in latency.

    EDIT: Theoretical gains and the latency difference versus practical or actual results might make this a moot point in the end but it's a interesting little hardware difference and AMD having a bit of a advantage for the Zen3 CPU's having taken this feature in mind would make sense though whether it ends up making a difference or not well probably a small one but that seems to be how SAM so far is working out and that could still change.

    Going to be interesting to see NVIDIA and AMD comparisons for re-sizable BAR support and different CPU's and all that but yeah this is a interesting little observation.
     
    Last edited: Nov 27, 2020
  19. Toetje583

    Toetje583 Member

    Messages:
    19
    Likes Received:
    10
    GPU:
    Vega 64 Red Devil
    Hello,

    Nice find I was experimenting as-well and however for me gigabyte is hiding those options if no 5000 series cpu is installed, however I enabled those options using UEFI shell after extracting the bios using UEFITool/ifrextractor and and looking for the correct VarID. I enabled them and decided to boot into to Linux results:

    https://i.imgur.com/I8iSlUc.png
    https://i.imgur.com/fZPF1PZ.png

    Without those options set the default bar stays 256mb however at least in linux the bar-size changed after enabling them, however sadly in windows there is no "Full Memory range" in my resources tab for my Vega64 :)
    So still unclear to me why older zen cpu's cannot use it in WIndows if linux obviously can.
     
  20. GSDragoon

    GSDragoon Master Guru

    Messages:
    399
    Likes Received:
    282
    GPU:
    AMD Radeon RX 6800
    Can you run any benchmarks with it on vs off in Linux? Maybe it's no improvement or slower on non-5000 CPUs.
     

Share This Page