Resizeable BAR support issues

Edit: The graphics driver version used matters.

Re-sizable BAR causes broken VRAM reporting and generally breaks games (low performance or start-up crash) with non-DX11 graphics APIs (DX9, DX12, Vulkan) with drivers that don't support it. With a RX 580 (Polaris) and 2700X (Zen+) on Windows 10, Adrenaline driver 20.12.1 works fine, whereas Enterprise driver 20.Q4 doesn't work (my report below is with the unsupported Adrenaline 20.11.1 drivers). On Linux, it seems to work fine on kernels 5.9 and 5.10 (openSUSE TW).

The quickest way to test if your driver works or not on Windows is to enable Resizable BAR in BIOS/firmware options, open Radeon Control Panel -> Performance -> Metrics. If VRAM is "Infinity", the driver doesn't support Re-sizable BAR. This is only applicable from the Radeon Control Panel (Windows Task Manager and GPU-Z will report a VRAM number regardless of the driver not supporting the option).



I can confirm Resizable BAR working fine with my 2700X, on an ASUS PRIME X470-PRO motherboard with 5833 Beta BIOS (Jan 15, 2021), and RX 580 with 21.1.1 Adrenaline drivers on Windows 10 20H2 as of 2021/01/21. Also looks fine from Linux.

______________

Original post:

Recently some Ryzen motherboard vendors released beta BIOS files (notably for older chipsets like X470) that adds support for Smart Access Memory (SAM), or specifically Re-sizeable BAR support. I have an ASUS PRIME X470-PRO motherboard that got such a BIOS.

With a RX 580, I noticed the following issues under Windows:

1. The VRAM usage reported by Radeon Software's metrics is "Infinity" (screenshot)
2. A game using DXVK (AoE2DE) flickers a corrupted white screen
3. Oculus Link doesn't work (flickers white upon starting)
4. Dota 2 with Vulkan had about 10 FPS (DX11 is about 80-90)

Specs:

• ASUS PRIME X470-PRO motherboard (5806 beta BIOS)
• RX 580 8GB GPU (20.11.1 drivers)
• Ryzen 7 2700X CPU
• Windows 10 20H2 (10.0.19042)

 

Thanks for report. For now, ASRock did not release BIOS update for X470 chipset based boards, just B450 ones. (Therefore I could not do testing, yet. Unless I pulled out of the box x570 board I have for friend's PC.)

Your post may help others when they start scratching their heads.

I have here RX 580 Nitro (non-Plus) which reports itself as 480 when low clock/poser vBIOS is enabled. (Clock matches reference RX 480.)
So, I would be able to test with similar GPU.

 

Wouldn't you need to enable these features in the bios and possibly whatever is in AMD 20.11.2 itself?



Bios should be something like this, guessing setting BAR to enabled would also set 4G to enabled whereas auto might leave it disabled it 4G is disabled.


Bios could have changed something though or the defaults could for whatever reason be enabled by default but I was under the impression SAM itself needed a bit to actually activate it.

Driver wise I'm not sure if there's a toggle or if it just works if the criteria is met (CSM disabled being needed as well?) nor if older GPU's are supported or not.


EDIT: Yep bios settings and 20.11.2 and other than 4G and BAR the compatibility support module (CSM) has to be disabled and all of that need to be set up for it to work, depending on hardware Re-Size BAR Support might not even show up in the bios.

Wonder if it's 1.1.8.0 I think perhaps the AGESA code and all it's modules or routines has a few issues that need to be worked out, newer chipset driver might help too for the software side of it after the bios is active such as the PCI Express driver unless the October 19th(?) release already includes this assuming the latest chipset software is installed on the system.



About how it looks like from my own Gigabyte bios though the BAR setting isn't there as I don't have a 5000 series Ryzen CPU.

 

Yep; the settings are in the BIOS, and on my ASUS BIOS, the settings are named the same thing (Above 4G Decoding and Re-Size BAR Support). CSM is disabled in my case also. The above 4G decoding option wasn't present in my BIOS prior to the beta BIOS update.

Outside of those issues mentioned, I don't know how to check if the Re-Size BAR support or the above 4G decoding options actually do anything beneficial on Windows. On Linux, it seems like something actually happens when adjusting those settings (reported it here on Reddit). I can keep that option enabled and things work ok from Windows, and Linux reports the full 8GB BAR size.

On Windows, even though Radeon Software's metrics has a broken VRAM counter, Task Manager and GPU-Z seemingly report correct information.

I'd be curious to see if Vulkan and games works on Linux along with VRAM reporting from AMD's driver with the Resize BAR option enabled, but I'd need to set aside some time for that one day (I think there's 3 different Vulkan drivers for AMD GPUs on Linux iirc). Another test I'm interested in is seeing if anything else breaks on Windows; I didn't test any other game or API.

 

SAM/Resizeable BAR is supported only on RX 6000 series by windows driver. It won't work on RX 580

 

Well, it apparently gets enabled and changes behavior.

 

It does work on Linux at least once these options were available, was trying to look up if there were any known issues but only suggestion was to keep a back up and be careful in case they trim older CPU support.
https://www.reddit.com/r/Amd/commen...agesa_bios_updates_enables_amd_smart/gdjqwpd/

From 256 to the full 8192 available with the 8 GB VRAM.

Which was from here when looking up bios info.
https://www.reddit.com/r/Amd/comments/k0oyhb/asus_x470_ryzen_5000_agesa_v2_pi_1180_beta_bioses/

Linux is it's own environment though and Windows is it's thing but it is good to see it working and not just for the newest GPU hardware although I'm not sure if that's the same on Windows though it could be the newer AGESA doesn't restrict anything.
(But the Windows AMD drivers could still have something.)

 

how does it change the results ?

 

Apparently: bold part in OP shows that it does something. While it comes with those undesirable effects. (At least for now.)
It means that even older cards than RDNA2 get it enabled. And that's exact opposite from Krteq's statement.

(But hey, it was apparent from context. All there was needed, was to read OP and very post I quoted together with my post.)

 

no,the results.
how much faster is it

 

For that, you are asking wrong person.

 

he should test more games
it looks like he's enabling something that's not supposed to be enabled.

 

and a 6800.

 

AMD stated that it will get enabled automatically moment compatible HW scenario is detected. No need to flip any options in OS/Driver.

 

With the calculations I did here, it seems like:

• BAR defaults to 256MB
  
• Presumably the BAR is limited by-default to that space if the BIOS doesn't advertise above 4G decoding
• If I vaguely recall correctly, this limit can increased from Linux regardless of BIOS support with some kernel option (I know I messed with the option in the past with a Macbook and eGPU)
• I recall Windows not increasing that BAR limit on my Macbook, preventing the eGPU from working with it (I needed to do something with DSDT modding to fix it)
  
• If 4G decoding is enabled in BIOS, Linux still starts with the 256MB BAR, but releases it, and allocates a larger size later on
• If 4G decoding and re-sizable BAR is enabled in BIOS, it seems the BAR is set to a large amount BIOS-side, and Linux goes with it (but it seems AMD's driver on Windows doesn't handle it well)

I'm not too sure what would be affected by a larger BAR, but it sounds like anything having high VRAM usage might benefit? poclmembench on Windows still reports the same numbers seemingly (205-206GByte/s).

In Device Manager under the Resources tab for the GPU, it has memory ranges; I don't know how these get set, but I have two theories:

• Those memory ranges are set at device install time, and stay that way until a new driver install (so any BIOS BAR stuff won't apply automatically)
• The PCI Lock option in msconfig would do something to affect that

Edit: The memory ranges in Device Manager do change based on the BIOS settings. PCI Lock option doesn't fix the infinity VRAM report with 4G and resize BAR enabled. Not sure how to convert the memory range numbers in Device Manager to something readable yet.

 

Here's the memory ranges (screenshots):

• Above 4G Decoding disabled, Re-Size BAR Support disabled
• Above 4G Decoding enabled, Re-Size BAR Support disabled
• Above 4G Decoding enabled, Re-Size BAR Support enabled (2nd page)

The memory ranges changed every time. When enabling the re-sizable BAR option, a new "Large Memory Range" label appears under Resources for the first memory range. So this seems to at least be one observation that the resizable BAR option does something notable technically.

PCI Lock is enabled for all those memory range screenshots (presumably in order to make sure Windows respects the BIOS ranges). Upon enabling this option though, both my motherboard's HDA controller and the HDA controller on the GPU don't start ("Insufficient system resources exist to complete the API."), but seemingly every other device starts fine; this happens regardless of the 4G decoding and resizable BAR options.

If the memory ranges can be converted to a readable size, I wonder if it would show the BAR sizes.

Seems there's all sorts of trees to look at from Device Manager pointing at different PCI bridges and resources, but Windows doesn't make debugging easy for this sort of thing.

On a different note, the kernel options for Linux I recall messing with in the past for a Macbook and eGPU was pci=hpbussize=10,hpmemsize=2M,nocrs,realloc (from this thread); might be useful info for something.

 

PCI Lock intentionally disables the ability for thte OS to perform PNP IO remapping.

 

Hmm maybe there's more to what AMD does with SAM or perhaps there's additional improvements with Zen3 here.
https://www.reddit.com/r/Amd/comments/k1zutr/rumor_regarding_support_of_sam_for_older_gen_zen/

Interesting.
Could still work on older CPU's probably will once AMD adds more support and how it all works out and what GPU's it gets enabled for in the display driver part of this little chain of requirements but that is a significant difference in latency.

EDIT: Theoretical gains and the latency difference versus practical or actual results might make this a moot point in the end but it's a interesting little hardware difference and AMD having a bit of a advantage for the Zen3 CPU's having taken this feature in mind would make sense though whether it ends up making a difference or not well probably a small one but that seems to be how SAM so far is working out and that could still change.

Going to be interesting to see NVIDIA and AMD comparisons for re-sizable BAR support and different CPU's and all that but yeah this is a interesting little observation.

 

Hello,

Nice find I was experimenting as-well and however for me gigabyte is hiding those options if no 5000 series cpu is installed, however I enabled those options using UEFI shell after extracting the bios using UEFITool/ifrextractor and and looking for the correct VarID. I enabled them and decided to boot into to Linux results:

https://i.imgur.com/I8iSlUc.png
https://i.imgur.com/fZPF1PZ.png

Without those options set the default bar stays 256mb however at least in linux the bar-size changed after enabling them, however sadly in windows there is no "Full Memory range" in my resources tab for my Vega64
So still unclear to me why older zen cpu's cannot use it in WIndows if linux obviously can.

 

Can you run any benchmarks with it on vs off in Linux? Maybe it's no improvement or slower on non-5000 CPUs.