Guru3D.com Forums

Go Back   Guru3D.com Forums > Hardware > Network questions and troubleshooting
Network questions and troubleshooting Does it work or doesn't it ? Discuss it here.



Reply
 
Thread Tools Display Modes
(Network) Storage - A lot of data
Old
  (#1)
Corrupt^
Ancient Guru
 
Corrupt^'s Avatar
 
Videocard: ASUS 1080GTX STRIX
Processor: Core i7 5820K
Mainboard: ASUS X99-A/USB 3.1
Memory: Kingston HypX 16GB 3Ghz
Soundcard: Essence STXII Senn. HD650
PSU: Super Flower 1200W
Default (Network) Storage - A lot of data - 01-04-2017, 15:31 | posts: 6,596 | Location: Belgium

Lately I've been reading around to get a NAS to store data on (more on that later) and more importantly (even more so than being able to attach it to a network), get some redundancy in case of a HDD failure.

Now I'm no stranger to RAID setups and NAS's in general, but the more I read about the limitations of conventional solutions, the more I realize it probably won't fit my needs.

What I'm looking for in particular:
  • Ability to store 20 TB of data with redundancy
  • Cost efficiency (20TB, HDD expenses basically)
  • Ability to scale to a larger size in the future (40TB, 60TB, ... growth is about 2 to 4TB a year)
  • Redundancy in case 1 or 2 HDD's fail (I'm looking at 10TB+ HDD's, see P.S. 3)
  • Restoration after a HDD failure: time to rebuild, etc
  • Read performance is more important than write performance

The reason why I'm asking is that with such an amount of data, I get the impression an old-fashioned NAS RAID setup will probably not suffice, especially RAID5 (which is bound to get an unrecoverable read error around ~12TB if I'm not mistaken).
http://www.zdnet.com/article/why-rai...rking-in-2009/

I have enough knowledge (provided I get some pointers to tutorials, etc) to work with Linux if need be and it made me wonder if there aren't any other solutions, other then classic RAID.

I've also read about Drobo, but the stuff I've read isn't all that good.

P.S. The data itself consists of generally huge files of about 1GB or more (video).
P.S. 2: Redundancy in case of a drive failure is more important then being able to read the data over the network.
P.S. 3: I'm aware 10TB HDD's are overtly expensive now, but I'm looking into this for 1 or 2 years down the road. Currently most of this data is backed up on a pile of 4TB HDD's in a hermetically sealed box, which is getting cumbersome.

Last edited by Corrupt^; 01-04-2017 at 15:58.
   
Reply With Quote
 
Old
  (#2)
nhlkoho
Ancient Guru
 
nhlkoho's Avatar
 
Videocard: EVGA 980ti sc+
Processor: i7 4770k
Mainboard: Asus Z87 Plus
Memory: Veng. Pro 16GB DDR3-1866
Soundcard: Soundblaster Z
PSU: EVGA 750G2
Default 01-04-2017, 16:23 | posts: 7,096 | Location: Washington DC

I'm getting ready to do this myself after I've outgrown my Qnap NAS. From what I'm reading, FreeNAS seems to be the best overall software for this. I'm new to the whole build your own NAS thing right now.
   
Reply With Quote
Old
  (#3)
Corrupt^
Ancient Guru
 
Corrupt^'s Avatar
 
Videocard: ASUS 1080GTX STRIX
Processor: Core i7 5820K
Mainboard: ASUS X99-A/USB 3.1
Memory: Kingston HypX 16GB 3Ghz
Soundcard: Essence STXII Senn. HD650
PSU: Super Flower 1200W
Default 01-04-2017, 23:16 | posts: 6,596 | Location: Belgium

Been looking at that one as well:
http://www.freenas.org/about/features/

Just trying to see what the best alternative (to classic raid) is.
   
Reply With Quote
Old
  (#4)
Hootmon
Maha Guru
 
Hootmon's Avatar
 
Videocard: MSI R9 290 Gaming 4G
Processor: 6600K@4.6/Corsair H115i
Mainboard: ASUS Z170-E
Memory: 16GB DDR4@2600
Soundcard: Soundblaster Z @ 5.1
PSU: EVGA SuperNOVA 850 G2
Default 01-05-2017, 19:04 | posts: 1,222 | Location: New Hampsha'

Once you go beyond conventional backup and/or RAID solution, you usually need an Enterprise-scale solution.

The problem with video files is that the data is likely already compressed in some way, and de-duplication doesn't work very well.
   
Reply With Quote
 
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump



Powered by vBulletin®
Copyright ©2000 - 2017, Jelsoft Enterprises Ltd.
vBulletin Skin developed by: vBStyles.com
Copyright (c) 2017, All Rights Reserved. The Guru of 3D, the Hardware Guru, and 3D Guru are trademarks owned by Hilbert Hagedoorn.