Functional 4-Way SLi PASCAL Titan X

[venturi]

Some critical updates for those technically minded individuals (all the members of this forum)
as far as i know, this is the first fully functional 4-way pascal rig where games and apps are actually using all 4 GPU's in SLi.






Current build:
Supermicro X10DRG-Q
2x E5-2699 v4 (44 cores/ 88HT) (3.7ghz turbo)
512GB ram DDR4 2400mhz ecc reg
(QUAD SLI) 4 Titan X PASCAL
2x Samsung NVMe 961 Pro pcie 3.0 (os drives)
10x Samsung 850 Pro SSD RAID (apps drive)
LG 31MU97z 4096x2160 true 4K rev C
modded P5 case, Noctua heatsinks
Digital power supply 1650w
Windows Server 2012 R2 Data Center, Ubuntu 15



I have functional quad sli using the new Titan x pascal cards.

The cards came with a back plate which acted as a thermal insulator, removed the back plate and heat dissipation is so much better. If you buy these cards remove the back plate immediately. The back plate is paper this aluminum with a plastic layer on the inside. Obviously it was just for looks at the price of cooking the cards
One can remove the back plate without making any changes to the front of the card





Detailed pictures and explanation to follow next:

I am only using the game benchmark as way of using a reproducible standard

So I'm using the built in 'benchmark c5l1" and "benchmark c5l2"


So here is the setup
Driver 372.54

Test material apples to apples

Painkiller Black
Havok physics enhanced mod multi threaded
FXAA - more than that is wasted at that resolution
Visual enhancement mods
Resolution - 4096x2160 (true 4k)


Running full eye candy Sli 4-GPU AFR 2 custom profile

From the bios shot you can see the layout I selected, please notice the what cpu goes to what pcie slot



Here is the view with the glass side off




Here are the bridge choices



Here are some of my selected manual bridge builds, these picture is only an example, not my final config



Important - my motherboard type -the x10drg-q is the only board type in the world with reverse order pcie sockets. The primary video card is the first socket on the bottom of the board, NOT the closest socket to the cpu(s) like all the other motherboards. Yes, it comes from supermicro that way.

this means that my sli pattern is completely different not just upside down since the order is reversed, opposite and upside down.

So
This is the score I can get with default brige - 303Fps is not bad at 4096x2160 with the fullest eye candy, havok mod, custom profile is the same for all tests running benchmark c5l1



This is with the higher v2 bridge (2 year old bridge) a 50fps increase, not bad



Same bridge installed upside down so that primary card has both connections, score only went up a few fps, but look at the minimum frame count go up
this is also because my primary video card with the monitor is the lowest one on my type of motherboard- which makes the primary card the coldest.



And various manual sli ribbon configs with all 8 sockets connected
Manual build layout running the more intensive benchmark c5l2 with greater physics

This is a custom sli bridge of ribbons layout using the metal connectors from three destroyed 4-way evga v2 bridgesThe metal connectors have grab pins on both sides for the card sli plug this makes the "upgrade to a better bridge.." exclamation mark go away


390FPS average, peak 1024FPS (game engine limit)



the "benchmark c5l2" is heavy on explosions and object trajectories, particles etc, so its a good test, as well of course as carrying the heavy mods for havoc and visual enhancement


...and honestly, --with full eye candy and physics, HD mods, almost 400 FPS-- at 4096x2160, isn't all that bad...

If I take the default painkiller black build and the custom profile without the additional eye candy/mods I get over 500fps


I have other games I have running in excess of 200fps at 4096x2160

(**but right now I'm having a tough time getting "No Man's Sky" running smoothly past 40-60 fps at 4096x2160 - the gane just doesn't run well at all, something just plain feels wrong with it, but that's another story.)



So, just on bridge config at 4096x2160, I went from 302FPS average, to 390FPS average.
here is an example of "any" game:







Thx

J

[Xanvast]

Not sure to understand, is it active/working outside of benchmarks ?

[fantaskarsef]

Hehe nice, what a beast
Good I don't have to dust it off though

[venturi]

Originally Posted by Xanvast

Not sure to understand, is it active/working outside of benchmarks ?

Yes
It's working

It's just that it's set up correctly, down to specific cpu sockets to specific cards and custom settings and specific sli bridge config on a reverse order motherboard

my motherboard type -the x10drg-q is the only board type in the world with reverse order pcie sockets. The primary video card is the first socket on the bottom of the board, NOT the closest socket to the cpu(s) like all the other motherboards. Yes, it comes from supermicro that way.

this means that my sli bridge pattern is completely different


A house of cards to get a good result

[Xanvast]

That was clear enough, but is it that particular pcie sockets order and bridge pattern that enables it ? Is it possible to achieve the same thing on a regular motherboard ?

[venturi]

Originally Posted by Xanvast

That was clear enough, but is it that particular pcie sockets order and bridge pattern that enables it ? Is it possible to achieve the same thing on a regular motherboard ?

I have not ever seen a single socket mother board feed 4 way sli to its potential, especially at real 4K and 5k resolutions with max eye candy and lots of physics.


At 2560x1200 and below a lot of configurations work ok because the demand is lower
Most rigs seem to be about the 1920x1080 rez

But to get 400FPS with max eye candy at or above 4096x2160, well, I have not seen it.

Anything over 120fps is a visual waste anyhow, but even 60fps at true 4K and 5k is tough for many titles/engines on just one card.
2-way sli is a good compromise if someone builds their own profiles, but out of the box the nvidia drivers have always been problematic if you're a perfectionist.

Anyhow, the point of the exercise is that nvidia for two months had made 4 way sli difficult / near impossible on pascal cards, but as of driver 372 it's been easier to set up again.

I had 4 way sli working on driver 369 with the pascal but had to do driver modding, with 372 it works without modding it


The other point was the bridge, all 8 connectors being used to get in the 400fps zone at 4096x2160

With 6 connectors for 4 cards, could only get to 350fps with v2 bridge, and 300fps with v1 bridges


The point:
So, just on bridge config at 4096x2160, I went from 302FPS average, to 390FPS average.

[XcroN]

Have you tried some games with 4 way SLI under pascal?

[tsunami231]

I thought 4 way was no longer support

[venturi]

Originally Posted by XcroN

Have you tried some games with 4 way SLI under pascal?

Yes, a little...

Fallout 4. custom sli profile, excellent
No man's sky. Tried nvidia and custom, poor results so far, very poor implementation for pc so far
Shadwen. custom sli profile, excellent
We happy few. custom sli profile, excellent
Hard reset redux. custom sli profile, excellent
Doom 4 So so, still trying to figure it out
Crystals. custom sli profile, excellent
Crisis 2 and 3. custom sli profile, excellent
Dying light. custom sli profile, awesome
Ethan carter redux. custom sli profile, awesome
Ashes of the singularity. Poor
Soma. custom sli profile, So so
Quake 4, doom 3, prey, So so, never ran sli well
Soma. custom sli profile, excellent
UT3. 32xAf, custom sli profile, Awesome
Unreal gold. custom sli profile, excellent
Ziggurat. custom sli profile, excellent
Singularity. custom sli profile, excellent
Sir, you're being hunted. custom sli profile, excellent
Quake HD. custom sli profile, excellent
Quake 2 custom sli profile, excellent
Assassins creed. custom sli profile, excellent
Batman Arkham. custom sli profile, with physics mod, good
Batman ark city. custom sli profile, good
Bloodrayne 2. HD \mod, threaded mod, custom sli profile, excellent
Necropolis. Custom profile, work in progress, good
Pollen. custom sli profile, excellent
SuperHot. custom sli profile, excellent
Solus project. Poor
Far cry 1,2, 3 custom sli profile, good/ok
Layers of fear. custom sli profile, excellent
The witches 3. ok 2 way sli, poor 3 or 4 way
The witcher 1 and 2. custom sli profile, excellent
Metro and metro 2033. custom sli profile, mediocre/ok
Outlast. custom sli profile, excellent
Valve engines- portal, hL2, custom sli profile, excellent
Portal 2- Very poor

Etc etc about 30 more titles

[venturi]

Originally Posted by tsunami231

I thought 4 way was no longer support

It works fine,
There is just a warning that it's not supported, but if you look at the pics it works well.

You can always make you're own profiles and you have 4 way sli

[Xanvast]

Originally Posted by venturi

It works fine,
There is just a warning that it's not supported, but if you look at the pics it works well.

You can always make you're own profiles and you have 4 way sli

I don't feel you gave me the answer to my question. I am interested by at least 3 Way SLI (coming off 4 Way Maxwell TXs) and I want to know if it would work with my setup or is it something that you did with the bridges, pcie slots order or the drivers that made it work in games. And could you please provide game benchmarks that show it's working ?
Thanks

[venturi]

An evga 3 way v2 bridge will work fine for your set up
It's 6 for 6 full connectors

Driver 372 and above supports it fine

I had maxwell 4 way tx sc and worked fine

As far as game benchmarks to show scaling, well the last long post shows the painkiller built in benchmark and it scales well on 3 and 4 way..


Otherwise I'm not sure what you're asking. Are you moving from 4 maxwell tx to 3 pascal tx?

[yasamoka]

Did you build this to play games?

Please no.

[venturi]

Originally Posted by yasamoka

Did you build this to play games?

Please no.

Lol, of course not.
Its for medical imaging, VMs, nuc med, isotopes.
I work out of my house sometimes.

The games or just a perk, my rig earns its keep first

[Xanvast]

Originally Posted by venturi

An evga 3 way v2 bridge will work fine for your set up
It's 6 for 6 full connectors

Driver 372 and above supports it fine

I had maxwell 4 way tx sc and worked fine

As far as game benchmarks to show scaling, well the last long post shows the painkiller built in benchmark and it scales well on 3 and 4 way..


Otherwise I'm not sure what you're asking. Are you moving from 4 maxwell tx to 3 pascal tx?

I already have 2 of the new Titan Xs but as they stated officially 3 & 4 Way are not supported anymore and will be disabled for games that's why I need solid proof it works before I consider droping another 1300€ or 2

[venturi]

Originally Posted by Xanvast

I already have 2 of the new Titan Xs but as they stated officially 3 & 4 Way are not supported anymore and will be disabled for games that's why I need solid proof it works before I consider droping another 1300€ or 2

The choice is up to you

I'm running 4 way sli

However
I would not embark on this journey unless:
1. You have a need greater than just gaming
2. You are willing to build your own application custom sli profiles
3. You have a motherboard and resources to feed the sli past 2 gpu sli



However

If you want solid proof
The best I can do is share my first hand experiences. And obviously the starting pictures to this post. I'm here to help, share, and gain back knowledge for things I don't know.

It may well happen that down the road sli will stop being used completely. But until a single card can power 8k Rez at 60+ FPS, think sli is safe

here are the driver 372.54 shots on 4-way Pascal Titan X:





I hope that helps.


The decision to purchase is entirely up to you

[Xanvast]

I have gaming needs that are not covered by 2 way Titan X SLI. I already have everything needed for the setup as I just got rid of 4 cards, I perfectly know what it takes. But the only proof you give is a photo of your monitor on a benchmark I don't know. The control panel still says it's only for benchmark mode.
And that's not solid enough so if you could show some more it would be cool. Otherwise thanks anyway for sharing your experience.

[Geryboy]

this 4 way sli uses the profiles for older games which support more than 2 way by default. The thing is games to come will not have that feature. Pretty sure you could 4 way sli older games on pascal.

I never made "custom SLI" profiles when I had it, surely they are limited to inspector and using existing profiles with flag alterration? that might not always work either, especially in dx12...

[yasamoka]

Originally Posted by venturi

Lol, of course not.
Its for medical imaging, VMs, nuc med, isotopes.
I work out of my house sometimes.

The games or just a perk, my rig earns its keep first

Phew. That's awesome!

You really need to watercool this beast.

[venturi]

My last rig was water cooled, but with the hardware I'm running there was very little I needed that for as I'm running close to room temp for most things

I like the elegance of water cooling, here are my last two rigs






Here is are my temps under a mild load in the current rig





And as for processing power to feed the gpus, here is my score on my 2 socket system

[XenthorX]

fps so much frames everywhere !

[ThirtyIR]

Venturi,

Long time bud! Your rig is an absolute monster!

I too have 4-Way Titan X Pascal - this is amazing if true.

Can you please run Rise of the Tomb Raider DX12 benchmark and post your scores?

I have 4-Way SLI enabled in NVCP but get the same "benchmark mode only" message but the cards do NOT work in 4-Way SLI in games - only in 3D Mark Fire Strike Ultra & Time Spy.

[venturi]

Originally Posted by ThirtyIR

Venturi,

Long time bud! Your rig is an absolute monster!

I too have 4-Way Titan X Pascal - this is amazing if true.

Can you please run Rise of the Tomb Raider DX12 benchmark and post your scores?

I have 4-Way SLI enabled in NVCP but get the same "benchmark mode only" message but the cards do NOT work in 4-Way SLI in games - only in 3D Mark Fire Strike Ultra & Time Spy.

ThirtyIR
long time


couple of things to try
1. make sure you have a 4 way bridge with metal connectors. The prongs on each side actually close a circuit and notify the card you have a higher band bridge.

2. make sure you're on driver 372.54, the 369.05 is broke

3. pic any game in nvcpl, go to sli mode under programs, does the drop down show 3 and 4 gpu afr 1 and afr 2 as a choice as in the picture I have earlier in the post?I should have 4-Gpu in the drop down.

4. with nvinspector, in base profile, sli mode, instead of auto, make sure you selected 4 cards. Apply changes.

5. I don't have Rise of the Tomb Raider. I get my games from GOG and never steam (don't like rent-to-play). I like clicking on the executable and bam being right in the game WITHOUT a third party ap constantly running in the background. Otherwise, you don't own the game, you're streaming it

here is an example of the profile from dying light:



I've had to get verve to get some steam games to run without steam. yes, I did buy the games, I just don't like the lie about "requires one time login/activation.."
such lies.

Anyhow,
what other games do you have and I can email you/pm the sli profiles for some and you can import them if you want.

I can look at RoTR and see if I can make you a profile to try


I had to make my own for medical imaging and the isotope software, so in the process, learned how to do the games as well.


(...except for No Man's Sky, this game has achieved a whole new level of suckage.
But that's another story...)



lastly:

I don't run windows 10.Never will,
So I don't have dx12.

My rig is on Windows Server 2012r2, running as a workstation, no issues from that what so ever.
So maybe in windows 10 there is a driver issue/difference that I don't know about.
Im using 372.54 without any issues or any changes in sli compared to when I had the regular 4-way sli with the original Titan X

I launch about half of my work stuff as a VM so windows 2012r2 was a good choice for that. Protected environment for a lot of stuff.

I also launch in no GUI mode when I have a lot of stuff to run. One of my apps is a unix only App so I use Ubuntu for that one.

I have had no issues in Ubuntu, but that is a different post.


At your service








and without 4-way sli, max eyec-andy/physics, it would be difficult to achieve

390FPS average, @ 4096x2160

[GhostXL]

Nice that it's working, but still...do you not need the new HB bridges to take full advantage of any Pascal SLI setup? Even if it's 2X, 3X, or 4X?

[venturi]

Originally Posted by GhostXL

Nice that it's working, but still...do you not need the new HB bridges to take full advantage of any Pascal SLI setup? Even if it's 2X, 3X, or 4X?

the best answer I can give you is this:

If you want it to run smoothly

The minimum bridge to enter the 3 and 4 way sli club for pascal is the metal connector bridges with the grab prongs on both sides of the metal connector.

The plastic connector gives issues but will marginally work

tested results:
Asus ribbons: conventional pattern, barely works, mixed poor results, bridge warning in driver

EVGA v1 4-way - barely works, mixed poor results, bridge warning in driver

Asus ROG 4-way - barely works, mixed poor results, bridge warning in driver

Asus Generic 4-way motherboard connector - barely works, mixed poor results, bridge warning in driver

EVGA Generic 4-way motherboard connector - barely works, mixed poor results, bridge warning in driver

MSI 4-way v2 metal connector plug with side prongs - works well, no bridge warning message

EVGA 4-way v2 metal connector plug with side prongs - works well, no bridge warning message


(best performance - all 8 plugs connected in special pattern with modified ribbons (4 of them) with metal interconnected plugs, no warning message and best results. I came up with this because my motherboard has SLI cards in reverse and upside down order. I have to put my 4-way bridges on upside down for example)


My recommendation

use the EVGA v2 4-way SLI bridge ($39)

use MSI v2 4-way SLI bridge ($29 --> 39-20 rebate)

and yes that is the best answer I can give you