www.overstock.com/Electronics/Mella...d-Refurbished/4851901/product.html?cid=133635 http://www.codemicro.com/store/product/BP/Ibm-32r176008-Refurbished.php Does anyone have any experience with Infiniband? If so is it easy using it for standard tcpip?
Read about Infiniband here: http://en.wikipedia.org/wiki/InfiniBand Its a low latency high bandwidth serial type interconnect not primarily used for tcp/ip it can be.
Infiniband is just like HyperTransport, and PCIe, it's just an interconnect standard. No desktop system I know has Infiniband connectors, they're usually found in servers or HPC clients. deltatux
It is like PCIe because you can multiply the links (in a single connection) for more bandwidth but its external and no it most certanly is not integrated into any desktop thats why im posting these cheap Host Channel Adapters, its mostly used in server clusters because the latency is so much lower than ethernet.
The connectors may be relatively cheap but the cables aren't... and most routers and switches don't come with Infiniband support either. deltatux
"InfiniBand uses a switched fabric topology, as opposed to a hierarchical switched network like Ethernet." Its nothing like ethernet u dont just connect a bunch of pc's to a router its a peer to peer style connection like firewire and mainly used for custom applications that dont use the slow tcp/ip stack, what i want to know is if i get 2 HCA's and a cable will it be easy using it for a p2p tcp/ip connection?
Im glad i could inform you about this technology i think more people should know about stuff like this used behind the scenes in supersomputers etc.
Answering my own thread it seems its easy to use ip over infiniband with IPoIB from openfabrics.org http://www.openfabrics.org/downloads/Windows/ the effective bandwidth of these adapters is 8Gbit and to compare the price of those Infiniband adapters to 10Gbit ethernet the cheapest i could find is about £400 per adapter! If theres any lan party maniacs out there Infiniand would make the ping times insanely low since the latency is so much lower than ethernet!
I think another problem that might be barring people from using Infiniband is the fact that most motherboards don't have an x8 slots ... and any free x16 slots run at x4 speeds which makes it less worthwhile than the card running at full x8 speeds. Of course some other boards may do x8/x8 but that means you lose multi-GPU capabilities. I think only the people with enthusiast class motherboards would appreciate this than the mainstream. deltatux
My 780i has 2 full x16 PCIe 2 slots and an x16 PCIe 1 slot why do those new mobo's only have 16 PCIe lanes?
I just ordered 2 of these: http://cgi.ebay.co.uk/ws/eBayISAPI.dll?ViewItem&item=290335539022 and a cable: http://cgi.ebay.co.uk/ws/eBayISAPI.dll?ViewItem&item=160589068855 For a total of £84.44 inc p&p, so lets wait and see if i can get the worlds cheapest 10Gbit network going!
You need an Infiniband switch. Not just an adapter and cables. I work with these cables every day as I build servers for Oracle.
You can, but you have to make one or both ends run a subnet manager, which takes care of setting up routes between IB devices; even if it's a point-to-point link, you still need to run a subnet manager. When I was looking at the same thing a while back, I found some talk on IB developer mailing lists (hosted on openfabrics.org) about developing a simpler way to set up these links, but did not find anything concrete. If you're running Windows 7, this person's experiences may help you get started: http://forums.overclockers.com.au/showthread.php?t=944153. The drivers he downloaded included a subnet manager service.
I did post this on post #9 its the well hidden download location for the OFED package does that have subnet manager software in it? http://www.openfabrics.org/downloads/Windows/ Oh yes I found the OpenSM thing in the package. I installed the OpenSM service and got the 4 missing files it wanted from here: http://openfabrics.org/downloads/Windows/v2.3/SymStor/ ibal.dll complib.dll winverbs.dll winmad.dll now the service just starts then stops I asume this is because I dont have the adapters yet.
Im having problems ordering the adapter in the overstock link on the first post on the international checkout when i pay with paypal it says the session has times out after i click pay now and if i pay by card it just gives me an error, any ideas how i can order the dam adapters? Oh and the adapters i ordered from ebay turned out to be HTX instead of PCIe and i had never heard of HTX i got a refund and didnt have to send them back tho! Ordered the adapters from overstock over the fone with DHL Express 5 day delivery expect to see benchmarks in at least a week.
Heres some performance numbers from my Infiniband network it was stupidly easy setting it up! ------------------------- Statistics ------------------------- Mode : blocking Operation Type : RW Number of Operations : 2048 Segment Size : 4096 Number of Segments : 2 Pipeline Length : 2048 Total Time : 0.1 sec Total Data Exchanged : 16.0 MB CPU Utilization : 99.87 Operation Throughput : 118411.72 ops/sec Bandwidth : 925.9 MB/sec Latency Arithmetic mean : 18.2 us maximum : 46.52 us minimum : 17.8 us Time Per Post 2048 posts without context switches Arithmetic mean : 0.20 us maximum : 2.22 us minimum : 0.16 us 0 posts with context switches Time Per Reap 2048 reaps without context switches Arithmetic mean : 8.20 us maximum : 195.3 us minimum : 0.62 us 926MB/s bandwidth and 0.018 milliseconds latency!!! Sadly the fastest file transfer i can get using a ramdisk mapped as a network drive is 196.6MB/s to the network drive and 126MB/s from it.
Pointless, for that price you could get a HP Procurve that does 1Gbit over here and that's already overkill giving you 1 or less ping. This is one of those "because I can" things rather then usefull for normal use.
I needed more bandwidth for backups over the network. Also this is no more pointless than your overclock.
