How much free space to leave on an external HDD used for storage? + Corruption multiple backups?

Discussion in 'SSD and HDD storage' started by 321Boom, Nov 21, 2017.

  1. A2Razor

    A2Razor Master Guru

    Messages:
    451
    Likes Received:
    42
    GPU:
    ASUS R9 Fury X
    Sometimes people include them in the filenames themselves, yep. Those hashes can also serve as unique identifiers for files, such as for database storage and lookup of them (kindof like an image GUID - global unique identifier). Viewing images is for all intensive purposes "good enough", but the human eye won't necessarily detect miniscule damages just like most people won't see or notice a quality loss with image compression (like jpeg).

    There's probably more post processing going on with the other ports for things like dynamic contrast, edge, or color enhancement. In the worst case modern TV's have motion interpolation too which adds seriously enormous amounts of delay. --Shame that the ports don't have any options buried somewhere in the menus, yet it's good that you've found the TV is doing some stuff differently on each.

    If you wanted to use it, you'd need a kernel that supports it on each device. Though that said for a home-network situation I doubt that multihoming would be much benefit to you. mptcp is more for if you want to combine multiple connections together, or get extremely high transfer speeds over a high latency link, or achieve things like "connection redundancy" (handoffs from one connection to another).

    --You may be surprised yet there's no official Windows mptcp implementation (just like there's no official Windows "SCTP" implementation either, heh). Closest that can be done is using a Linux or BSD VM as a router run on the same machine (router inside computer, for the computer itself). There's some OpenSource projects doing just that as a short term solution to get mptcp on Windows... Apple is pretty much at the forefront in pushing mptcp (for global adoption), probably mptcp will be in the Linux and BSD mainline kernel branches before Microsoft gets onboard. (really ironic considering Microsoft is generally thought of as a technology leader)

    You'll find mptcp in quite a few commercial "Internet Bonders", yet the largest scale deployment is Apple's phones and servers. I'd personally have thought that Microsoft would have more interest in leading Internet protocol innovation ... though they're showing very little interest in doing so.

    Yep, you got it, as things are your clients "so far" are Windows machines.
    -The "d" at the end of OpenSSH there stands for "daemon". The OpenSSH project is actually both a client and a server (daemon), and you'll find that some Linux distros will throw in that "d" at the end of the service name to avoid confusion [make it clearer that it's talking of the server-component and not the client]. You can also install the SSH client and server independently, but the core openssh package tends to come with the collection of both together (more or less the complete solution / all tools you need to start a server and communicate with it).

    Windows, Linux, Android, iOS, you name it and there exists an SFTP client for it. Very widespread use, pretty much becoming a defacto standard for secure file access, even some webbrowsers getting support built in to them for it.

    NetDrive being commercial limits what OpenSource code they can "legally" use. OpenSource projects like WinFsp (which are basically FUSE' concept ported to Windows) are GPL.
    --In jist, unless a project is "L"GPL (lesser GPL) -- like "Dokan", then any works that link the GPL-code (dynamically, statically, doesn't matter -- use of that code in any way) automatically also become "GPL" (aka, public domain). In otherwords, source has to be released which means that it's difficult to protect your assets. Some licenses are more restrictive than others, but in general unless a library is something like MIT or BSD license, most companies will be pretty weary of touching them.

    ^ The desire to keep source "closed" isn't necessarily a bad thing, though it can mean reinventing the wheel when there's projects (like WinFsp) out there that already work. NetDrive is pretty stable, but it's a different project completely from the ground up.


    Anyway, getting back out of licensing those softwares all work similar (have same end goal). Whichever you pick and configure will mount an SFTP share as a 'volume' on the machine. That means you configure them once and they "do their thing". They provide what looks like a disk-drive to anything you use .. be that Windows Media Player, VLC, drawing programs, a text-editor, or Windows Explorer.

    Yes, ahem, this is for ease of storing files that you're going to work on when you're at home. Convenience, quality of life: yeah, that, maybe. (COUGH)

    "These third party clients are WinSshFs and NetDrive?" -- there's actually a ton and ton of them as you'll see when you search around, but yeah, they're all the same core concept. Mounting some remote-share with various protocols that Windows doesn't natively support.

    --It's possible to use Windows Explorer like-normal with those two clients.

    FileZilla and WinSCP are standalone programs that can transfer and synchronize files to servers. They don't mount the shares as a volume / drive, and they won't give you access to those network shares from other software.

    --They're more useful for infrequently accessed shares. For instance, say that you have a website that you might want to upload files to "occasionally", then it just may not make sense to leave mounted and you might prefer a solution like those tools.

    I'd say that the best bet is an automated scan with everything you can. If you have a machine with Avira, have that scan your files. If another machine has Malwarebytes, try scanning everything with that too, same goes for anything else including even Microsoft Security Essentials or ClamAV on the NAS itself. The more you use, the greater the chance of detection if some infection were to somehow sneak on there.

    --It'd be a good idea to tweak your AV settings in all cases because we don't necessarily know what the AntiVirus programs will recognize "as a network-share". For instance, it's always possible that they'll add detection for the third-party network mounting tools as well as Windows' native build-in mounting.

    The server itself would be fine (not infected), yet malware could still do damage ... like replacing your images with pictures of squid, infecting stored executables on the NAS, encrypting files (randsomware), etc. The more you can restrict access (eg, read-only), the safer you'll be in this regard since there'll be less files that an infected machine could touch or overwrite.

    See above, though to elaborate a bit:
    --Storing malware on the server alone does nothing beyond that [just copying files to the NAS doesn't infect the NAS or other machines necessarily] ... though any computer that can write to the NAS can also destroy data on the NAS (delete, edit, etc). This is why you'll want to give some thought to which machines should have write-access.

    The game machine only needs limited write access for transferring off videos. A media-PC would only need read-access to play movies, videos, and so on. As long as you can restrict access like that, and only do questionable things on machines that have restricted access, then your data will be very safe.

    Yep! You can do this on FreeNAS, or you could run it from Windows on your mounted volume (may be faster from the server over SSH -- due to no network-chokes). FreeBSD comes with everything you'd need to generate and check hashes for files. If you look around you'll find that alot of people have written "bash-scripts" and "one-liners" using these tools to loop over everything in a directory and write out or compare hashes to / from files.

    Example shamelessly ripped from StackOverflow (boards) of a one-liner generating hashes for a directory:
    Code:
    find ./path/to/directory/ -type f -print0  | xargs -0 sha1sum
    EDIT: (description of the above)

    Realize that this may look intimidating, but there's actually not much to this.

    Code:
    x:~$ mkdir test
    x:~$ cd test
    x:~/test$ echo "test" > 1.txt
    x:~/test$ cp 1.txt 2.txt
    x:~/test$ find ./ -type f -print0
    ./2.txt./1.txt
    x:~/test$ find ./ -type f -print0 | xargs -0
    ./2.txt ./1.txt
    x:~/test$ find ./ -type f -print0 | xargs -0 sha1sum
    4e1243bd22c66e76c2ba9eddc1f91394e57f9f83  ./2.txt
    4e1243bd22c66e76c2ba9eddc1f91394e57f9f83  ./1.txt
    
    Code:
    **ripped from the documentation**
    
    -f      regular file
    
    -print0
          True;  print  the  full file name on the standard output, followed by a null character (instead of
          the newline character that -print uses).  This allows file names that contain  newlines  or  other
          types  of  white space to be correctly interpreted by programs that process the find output.  This
          option corresponds to the -0 option of xargs.
    
    So what is all this? Well, in English version:

    mkdir test -- make a directory
    cd test -- move to that directory
    echo "test" -- writes "test" to standard output (to the console)
    > 1.txt -- pipe the output from echo to a file instead (creates a file containing the line "test")
    cp 1.txt 2.txt -- copy 1.txt to 2.txt (now we have two files in there, hooray)
    find ./ -type f -print0 -- sometimes when you want to see what you're doing it's easiest to execute it in 'parts'

    find is a command line tool to search files and directories. In this case we want to find files in "./", since that's the folder we're in right now (that newly created "test" folder).

    We get a single line output "./2.txt./1.txt" -- there's actually a character between those two that we can't see represented in the shell here, a "\0" (usually called a null terminator). That's being used instead of a newline delimiter because we asked for it with "-print0".

    "|" -- is used to feed the output of one program to another.

    " | xargs -0" -- xargs can take that null-terminator delimited output from "find" and make them in to spaced and quoted (if needed) command line arguments (basically make them ready to pass to another cmdline tool). -0 denotes that null-terminator will be used instead of newline like in "find".

    This output looks like:
    "./2.txt" "./1.txt" (two nicely formatted arguments to pass along)

    find ./ -type f -print0 | xargs -0 sha1sum -- So now all we need is to pass these in to a tool that generates hashes and takes a list of files. (like sha1sum, but could be sha256, md5, or any other tool that takes similar arguments)


    Everything seems like a flood at first, but there's also only so much that's out there too. When you first used Windows or DOS, it probably took some time getting used to as well. It's really much the same as picking up any new game, where the controls feel overwhelming and there's so many new systems and mechanics to learn. After a bit though, that all calms down when the realization hits that you're getting closer and closer to the ending.

    Playing with your file server, BSD, Linux, or any OS is much the same as playing a new videogame. Once you get the mechanics down, everything gets easier and easier with the more you experience.

    No problem as always, and you too!

    Also sorry if this seems that I'm basically preaching switching away from Windows at times with all this OpenSource stuff. I'm not even that strong an OpenSource advocate, but moreso it's just a matter of stepping out of that comfort zone just a little-bit and seeing all this awesome stuff that's offered out there (especially in the server space). Windows has its place, and so does BSD, and Linux, Android, Mac, and everything else built on them thereof [and especially OpenSource where commercial works just wouldn't happen]. They all have their strengths and things they're awesome at if you just push past the initial learning curve and see what they can do for you.

    It's like having a chocolate cake, vanilla cake, and so on. All are great, and having one flavor alone is great, but more is always better (no reason to limit yourself to just one). :)
     
    Last edited: Mar 31, 2018
    321Boom likes this.
  2. 321Boom

    321Boom Member Guru

    Messages:
    111
    Likes Received:
    10
    GPU:
    GTX980 Ti
    Thanks for confirming that. Yep, you're definitely right about the GUID part, there were times when I needed to search for the same image again (to find out who the artist was), and sometimes googling just the hash brings up the same image even from different websites that are hosting it. (The image site I save anime art from used to only use the hash as the file name before, about a year ago they updated it so the file name includes more information like: the anime/game it's from, character's name, and most importantly the artist's name, and the hash.) Here's what I'm getting at with this, all anime art I'm saving now comes with the artist name, so it's easy to find out who the artist was (so I could visit their gallery and see what other art they drew), but I've got a HUGE bunch of art which only has the hash as the file name, so finding the artist isn't as easy. As I said above, googling the hash sometimes will find the image again, so from there I could find the artist, but anything I can do to find the image again in the cases where the googling the hash won't find anything? (maybe there's some program or website that combines the hash with the GUID database or something)

    Btw, if the hash is those numbers (2c35d9e70306b3696fed52eef483e259) tacked on at the end of the file name, isn't it too short? I mean, doesn't MD5 use 128 bit, SHA uses more, so the hash should be longer? (the hash is only 32 characters), or are each 4 bits represented by 1 character? (sorry if I'm mixing things up like if bits have nothing to do with character length)

    Got it about the human eye can't detect slight signs of corruption, so it's definitely worth making a database of my hashes periodically then.

    Yes I agree, on the PC HDMI port the colours are a slightly more matted out (kind of like when you switch on Game Mode on a modern TV with a console, this also dramatically reduces input lag), so it shows that most of the 'image enhancing features' are switched off. Small trade for such a huge decrease in input lag. Yep, I read somewhere that the PC port makes the TV 'act' like a monitor rather than a TV, that's why it's so beneficial having a gaming PC hooked up to that port instead.

    Wouldn't mptcp provide a better checks than regular TCP though as we said in post 97? (so it does sound beneficial :/)

    By 'kernel that supports it on each device', you mean I need to keep this in mind when purchasing parts, or it's just something I could install later? (some way to update the kernel etc)

    Haha I know what you mean about Microsoft and the irony, for a company thought of as 'the technology leader', they sure do have their shortcomings.

    Right, thanks for the explanation. So I need to use both OpenSSH and SSHFS, or will they both be doing the same thing? Is OpenSSH built on FUSE also, or am I mixing something up here?

    Hmm thanks for the in depth explanation about licensing. Seems like lots of red tape, the more source is open, the more people could continue adding onto it/pitching in to make it better. No need to 'reinvent the wheel', yet I understand how this could be good and bad, because building something from the ground up could result in something new/innovative/better than the current options.

    So WinFsp and Dokan are another 2 alternatives to WinSshFs and NetDrive?

    'They provide what looks like a disk-drive to anything you use', in my case this will be one big 24TB volume, correct?

    Haha, of course it is :p

    Got it, that's awesome that they provide various protocols which Windows doesn't already have, and also that I'll be able to continue using Windows the way I'm used to :)

    Oh ok, got it, I don't think they're particularly something I need for my intended use with the server then? (I need access to it constantly.)

    Got it, so as many as possible :p Good to know it will be scanning both from the Windows side, and also on the BSD side.

    Noted about the network scanning features. Thanks for the heads up.

    Ok that is VERY BAD, how could you call that 'the server will be fine' lol. (it's purpose is storing data after all :p)

    2 questions regarding your quote 'The game machine only needs limited write access for transferring off videos. A media-PC would only need read-access to play movies, videos, and so on.'
    1. If the gaming rig had to be compromised, and it only has write-access to one folder (where to upload my gaming recordings, I'll move them around and organize them with the ECC desktop from there) it cannot infect any other folders? (i.e. the malware, virus, etc can't spread to the other read only folders?)

    2. My 'media-PC' is going to be my ECC desktop. I can't have that on read-only since I'll be using that as my main desktop (saving art, updating spreadsheets etc). Opinions/care to elaborate on the matter?

    Damn dood, you sure know your way around this stuff, that code was like x_X for me. Thanks for the in depth explanation of the alien language, I really appreciate you taking the time to explain it, I understood like 70% of it (thanks to your English version), but I really hope I don't have to be doing that stuff manually and could just download a one liner lol. 'Realize that this may look intimidating, but there's actually not much to this.' Lol not much to it for someone that knows what he's doing, not a newbie haha :p

    Got it, so through FreeNAS is best to avoid network chokes then. Any idea how long this hash generating will take? (i.e. long hours like a verify?)

    One thing I really didn't get, in your English version, 'cp 1.txt 2.txt -- copy 1.txt to 2.txt (now we have two files in there, hooray)', why would a copy of the file 1.txt to 2.txt be necessary, aren't we just generating hashes, so why would any copying take place?

    Haha it's slightly different from a new game, games come with nice fancy GUIs, not code :p Jokes aside, yeah I'm sure it will get easier down the line, especially when I'm implementing everything and getting to grips with everything firsthand rather than just reading about it.

    Nah don't worry about it, I completely agree with you about the whole Windows thing. As far as ease of use goes, Windows is definitely the easiest, but then there's so many things that are half-assed on Windows that I'm willing to venture to the other side, especially if it means keeping my data safer. After all, it would be a real shame having something as beautiful as a 24TB RAID10 system, but not having the correct and best measures for it :) (like ZFS, SFTP mounting etc.) Thanks again for all of your time in walking me through it. I really can't stress it enough, I understand everyone is busy with their own lives too, so I appreciate your constant support/mentoring.
     
  3. A2Razor

    A2Razor Master Guru

    Messages:
    451
    Likes Received:
    42
    GPU:
    ASUS R9 Fury X
    -Tried Google's "Search-by-Image" service, instead of searching via the Hash? Of course this assumes you still have the base images that you can re-upload. Google's search database is probably as good as you'll find far as doing this.

    -Each hex character is something called a nibble, 4-bits you're right on that part.
    -The smallest addressable (memory or file) unit on most computers is "a byte" (without using bit-shifting), or two nibbles / 8-bits. That means we could store a 128bit hash in (128 / 8) = 16 bytes.

    Unfortunately you'll notice that I called this 16-bytes and not 16-characters. Certain values of a byte, even in ASCII coding do not have a visible character assigned to them, or may have special-meaning (like 0x00 -- null terminator, newline, return, tab, break, etc). One way to get around that is to go byte by byte across the 16-bytes of the hash, and double them, converting each to a nibble-pair and then its hex-digit representation. This guarantees that there's no overlap with special characters, and that each is friendly / representable in any character-set (human readable & string storable). The null-terminator typically marks the end of a string, so that'd be a no-go if the hash contained a single byte of value '0'.

    mptcp only adds support for multiple connection endpoints and changing endpoint mid session. It doesn't add any additional checks and otherwise functions similar to TCP, which is why Internet gateways don't need extra support for it (don't have to be updated / works without change other than to the endpoints). Even a lowly consumer router NAT translation works with mptcp subflows without change (as they get treated as regular TCP connections), but mptcp was more or less designed to be this way (work with minimal changes).

    Operating System kernel. eg, Windows, Mac, Linux, FreeBSD. -- All purely software, and if Microsoft got onboard would likely just come in through a Windows Update down the road. (in however many years it takes for adoption)

    It'd be possible to implement on Windows without Microsoft getting onboard through a userland TCP stack implementation + a winsock LSP or WFP. I'm not aware of any efforts to do that (probably because it'd be really-hard). All in all it would be much-much easier if Microsoft did so themselves. To do it in user-space would be kindof like building a socksifier, only you don't use socks and instead pipe the Windows TCP-stack in to your own "built in usermode" TCP stack (yo dawg, I put TCP in your TCP so you can....), which then handles the connection using something like winpcap or a custom bridge-driver. You essentially wouldn't replace the TCP stack and would leave it there.

    Networking, yep that's a huge shortcoming of Windows. When people say that Windows isn't a good choice for servers, there's actually alot behind that other than MS hatred, cost, or favoritism.

    --Best example that I can think of is interface selection and routing rules. Say that you have two Internet connections on Windows, two-routers, two separate gateways. How do you specify which gateway should be used for an address you want to connect to? Can software even do this?

    Windows has a routing table (emphasis on "a") and you can add rules in here based on a destination IP. While that works, what if you want two connections to the same server, one out of each of your routers? Well, you just can't easily do that (without two IP's), there's "tricks" that can achieve it ... yet it gets ugly pretty fast. Linux or FreeBSD on the otherhand an application can just specify which interface it wants to bind for outbound connections "per socket". The OS's can have multiple routing tables too, one per interface, whereas Windows follows a single routing table premise.


    File-Systems and data-storage is also quickly becoming a Windows "weakness" (comparatively speaking). Even Microsoft Storage Spaces (which admittedly is a big-improvement) is still quite feature lacking compared to the choices available on the Linux & BSD side.

    OpenSSHd you'd use as your "server" that SSHFS connects to (as a client). Only the client implementations (SSHFS) are built on FUSE, because only the clients mount a network drive. The server doesn't have to mount a drive, as all files are local to it (it's just sharing them).

    The gaming machine's uploading of videos "might" be a good candidate for using those types of "one-shot" tools, since access is just going to be getting games (probably) copied on to the SSD (unless you're also going to run games off the NAS, that's cool too), and uploading your recordings. It's also much less likely that malware will try to self-propagate this way if the drive is left un-mounted. Since then to copy itself to your file-server, the malware would have to understand how to make a connection to it, eg -- not as simple as just writing files to disk [would need to get credentials to the server & have a built in SFTP client within the malware].

    Well, it all depends how locked down your access rules are for each machine. :D

    --If you only have read only access, like on a "dedicated" media-PC.. Can't write files, can't delete files, etc, not much the infection can accomplish other than destroying the media PC. So, no-problem, impossible for infection to spread itself, probably also for it to get at sensitive data as I assume there's nothing sensitive in Anime downloads or movies.


    The way I personally think of it is that your NAS is not a backup in itself. There's a common saying that you don't actually have a backup unless you have your data in "at least" two physical locations. The NAS takes care of many causes of data-loss, but malware falls more under user-error, since it's not hardware failure.

    Only true way to be completely-thorough is to be smart with your access-rights on the NAS and to also keep external (disconnected) backups on external drives. The last-resort defense. A shadow-copy (versioning) of files on the NAS can "kindof" protect against malware, but the NAS even then doesn't know the intent of the user, and you'd have to by-hand catch malware tampering with files for that to be useful.

    ----Easier said than done, takes some careful thought. Definitely will be more annoying than just giving yourself full access on all machines, though in the end very-worth-it.

    I'm presuming that malware is only interested in tampering with executable-files and not media-files. So, if all you're uploading is media-files, yes, you'd be pretty safe / the infection would be stopped at the gaming PC.

    --Of course, don't quote me on that since there have been "bugs" inside "Media Players" (I'm looking at you Microsoft, remote-code exploits) and also bad macro features (COUGH, Microsoft) that allow stuff such as "opening a web-page" embedded inside an MP3 file.... How anyone can think that's a good idea is beyond me.... Then again, we have all that garbage like MS Word macro-viruses too.

    In this case I suggest that you take extra-precautions on media-playback, especially if you use Windows Media-Player (**see above**). I consider this a low-risk personally, yet I'd get yourself a copy of Sandboxie and isolate whatever software you use for playing content (be that VLC, WMP, etc) inside a sandbox. You probably also should use Sandboxie to further secure email and web-browsing from each machine, but that's another topic.

    Hash generation and comparison are the same-speed as eachother (since both involve calculating hashes for each file). Amount of time that it takes is usually the amount of time it takes to read-back the files off the disk (choked by read-speed). So, it really depends on how much you want to verify. To verify hashes on every file on the drive implies needing to read all files on the entire drive, and that could be enormous. Meanwhile just verifying a few games might take a minute or less. Big-files will also take less time than many small files due to this being hard-disks and somewhat slower at random access vs sequential.

    If you generate hashes remotely (not on the NAS), you'll hit a network choke before you hit the drive IO choke most of the time. Though again this might not even matter because you're not running a comparison on very much.

    With only a single file, we could just run sha1sum directly without using find. Need at least two files that we can show that it goes over all files in the directory, or really just to show what the purpose of that find command was in the one-liner. ;)

    ^ This seriously.

    Being the captain' and immersing yourself in it will speed learning up tremendously, just like learning any spoken language. It's also much more fun once you get to start throwing drives in a machine and have something physical to play with. (then it's like having a toy, not studying, and the FUN [tinkering] begins)

    Windows has proven itself fantastic for the user-experience, the GUI, and "just-working" -- definitely user-friendly is something they're winning in (big-time, market share shows such).. At the same time though, you learn pretty quickly when you look at the server-side just how small Microsoft's presence has gotten there (how it's a complete 180 and a fraction of what Microsoft's share used to be). Linux pretty much took over the entire server market.

    Ironically in ease of use, you may find that Linux servers are actually easier than doing similar setups on Windows. It seems counter intuitive, but the market generally picks a winner not just in cost alone. It's kindof like when you watch "Who wants to be a millionaire" how almost always ask-the-audience turns up the right answer for how to do something, heh. But yeah, collective opinion polling by just looking at consumer choice is usually pretty telling.
     
    Last edited: Apr 12, 2018
  4. 321Boom

    321Boom Member Guru

    Messages:
    111
    Likes Received:
    10
    GPU:
    GTX980 Ti
    This is exactly what I was looking for! Thanks so much. Yep, I still have the base image/s, I just need to bring up the page I saved it from again, because there will be a tag of who the artist that drew it was on the page, and from there I could find his full gallery :) You really have no idea how much this is going to help me out. Thanks so much again.

    Awesome, thanks for the detailed explanation :) Very interesting Just one thing I didn't get 'The null-terminator typically marks the end of a string, so that'd be a no-go if the hash contained a single byte of value '0'.', By null-terminator you mean 0 correct? If 0 marks the end of a string, shouldn't it be the final character in a hash? Why are you saying it's a no-go? (There are 0s in the hash I listed: 2c35d9e70306b3696fed52eef483e259)

    So basically I'm stuck with regular TCP then for download verification? Nothing better to ensure what is getting saved from the ECC desktop directly to the NAS (as the default download location) makes it there safely?

    That does sound difficult :/ Guess I don't need to take note of any of these options since mptcp wouldn't be beneficial to my use, or is Winpcap something I could utilize?

    Thanks again for another explanation and example. While that example won't apply for my use, I could see how Linux or BSD could be superior in this aspect. I too would prefer another choice instead of Windows to handle all my data, I've read and heard that it is definitely not the safest route for servers from many sources.

    Ok, so I got that OpenSSHDd is on the server, but if SSHFS is for the client pcs, and my client pcs are going to be Windows machines, I'd need the Windows implementations of these like WinSshFs, not the actual SSHFS? (SSHFS doesn't work on Windows, it's for BSD and Unix right?)

    Hmm that does sound beneficial, especially the protections from malware, remember that the gaming rig will rarely be browsing any websites though, only times will be to update drivers or programs. So will it still be at a risk of malware/viruses? Here's something I don't fully understand, to get infected with malware or a virus, do I actually need to go into a malicious site, or just having an internet connection is enough for it to come in cause it has a 'door' to use if it needs to?

    Unfortunately I'll be connecting the gaming rig to the server more times than I originally thought. Remember when I told you I did the testing around with the HDMI ports, well anime art looks nicer on the Full 0-255 range which the gaming rig is on, so although I'll be saving the art with the ECC desktop, when I actually want to sit down and enjoy viewing it, I'll be doing that from the gaming rig. So, some questions about this:
    1. If the gaming rig is just viewing them (these folders will be set to read-only for the gaming rig), no worries that it can corrupt something while viewing since it's non-ECC right?
    2. Is there some way I can connect the gaming rig to the server with without having an internet connection where something malicious could come in from? You know maybe have my own intranet or something? LAN? Directly by UTP?
    3. If using FileZillla or WinSCP, could I also view my anime art and some gaming videos with it as if I'm using Windows normally? Or are they just to move data across from one place to another, and not function like Windows Explorer?

    There are lots of points of very good advice in this quote. I fully agree on most of the things you mentioned here. While it will be more cumbersome having to restrict access, it will be worth it for integrity's sake and safety.

    Definitely have regular offsite backups, those will be offline for most of the time so malware can't easily tamper them, and they'll be the last line of defense as we like to call it.

    I'll put more thought and careful consideration into what machine has access to which folders once everything is set up, cause yes as you stated, although annoying, very worth it.

    That's good to know. So even if I run virus and spyware/malware scans on the recordings before I upload them to the server these 'bugs' can still go undetected?

    One other thing to note, apart from gaming recordings, I'll be backing up my games into the server once the game is complete. Can't go for symlinking anymore now since I have to keep disconnecting the internet connection before gaming due to the Meltdown/Spectre updates performance hit. Games are executable-files correct, so malware could tamper with those if not the recordings/media-files, or is it the same risk and games are a different type of executable?

    No I don't use standard Windows Media Player. I use CCCP (Combined Community Codec Pack which uses MPC-HC (Media Player Classic Home Cinema)), VLC, or rarely PotPlayer when VLC has trouble with certain files (usually something in HEVC gives me trouble on VLC).

    Sandboxie sounds interesting, and you had also recommended this program before. Will I need to use Sandboxie only for media playback, or also when viewing anime art that I already have saved, saving new anime art from the web, updating spreadsheets etc? If it creates an isolated session, will it still manage to save the art to the default download location in the server? (since it's isolating itself from it?)

    That could take quite a while then, even on my current 7TB of data, let alone once the server starts filling up more and more! Well I guess I'd want to generate a hash for everything I have, makes the most sense right to make sure everything that is stored on the server stayed in 100% integrity as when I originally saved it.

    Got it, so no actual copying will be done when I'm running these one-liners on all of the data, this just happened due to the example being only one file?

    Agreed :) What you call FUN though, I call heart-wrenching terror, especially if it's in code lol, and especially till I see that I'm happy with the end result :p But you put your mind to it and start getting to grips with it. This was the same (but simpler I think) when I got into recording gameplay, at first it's x264 vs using the GPU, then all of the different features like Constant or Variable framerate, CRF (quality) and x264 encoding preset speeds (veryfast, superfast etc), Lanczos vs Bilinear vs Bicubic, etc. Was a lot to take in, and lots of testing and experimentation to be done to get the desired quality of recordings I have now, but it was all worth it in the end, and I'm sure that the server with ZFS and RAID10 (love saying that) will be sooooo worth it xD

    Indeed it is, as we said although it has it's shortcomings it's still an amazing operating system for lots of other uses. Yes I agree with you on how small the server share is with Windows, which is not surprising though when it lacks some of the nicer features Linux has.

    Lol, this is easier??? We've skipped a 100 posts talking about the subject :p Interesting to know, I thought people picked Linux more due to the extra features which Windows does not have implemented. Yep, collective opinion, the tried and tested method, all plays a part into what people will choose and opt for.

    Thanks once again for all the help and information, especially that Search-by-Image from Google, not only are you informative on data, but also on many other workarounds.
     

  5. A2Razor

    A2Razor Master Guru

    Messages:
    451
    Likes Received:
    42
    GPU:
    ASUS R9 Fury X
    > By null-terminator you mean 0 correct?
    Yep, a zero. The value 0, and not the character representation of "0".

    > If 0 marks the end of a string, shouldn't it be the final character in a hash?
    Final character of every string is typically a zero (a terminator). However ... the hash itself might contain a zero for a byte (IF it was byte-coded), thus why the hex-representation is used instead. [eg, to permit storage inside of a string -- for a text-file, on a website, and so on]

    While that "0" (as in the character "0" in a string) is legal in a hash (hex-digit zero), the representation of "0" won't actually be stored as the byte value zero inside the string {0x00, \0}. --This may sound a bit confusing at first and so it may help to look at an ASCII table or the UTF-8 encoding (of numeric values specifically). One thing that's universal of all text-encoding is that null terminator, value 0, is reserved / special meaning and this makes it hard to just store bytes in a string (unless the string is fixed-length / doesn't use a terminator).


    --If that was a string containing "just the hash", it might look like this "2c35d9e70306b3696fed52eef483e259\0" with a NULL at the end.

    Yep, as long as you're downloading using regular HTTP then you're relying on just TCP's packet checksum. Sadly this has not changed very much over time ... we've just been relying on stored hashes on websites, even today. Even under the hood of game downloaders, you'll find that most of them just use "HTTP" for the game's downloading. However, at least now some of them automate hash-checking and have a verify feature to make sure the content is intact.. --Yet ultimately this checking is no different than tools like MD5Summer in functionality (they just are fetching an extra hash-file [again over just "HTTP"] which contains hashes for each downloaded game-file)

    From the winpcap site:

    --winpcap is more of a library for developers than a standalone tool. You'll find it used in software such as Wireshark (packet capture / analysis tool), but it could also be used to implement your own TCP-stack. pcap provides an easy way to "send & receive raw packets" from a user-mode application.

    Right, you'll have to look into either win-sshfs or the commercial products that do similar. The main SSHFS project won't run under Windows (Unix like OS's only), and there's no ports of it for Cygwin either to my knowledge.


    This one is an "it depends". If your Windows machine is locked down, all ports restricted, not running software that maintains any connections to the outside (which means auto-updaters all OFF), firewall is ON / no inbound connections are allowed AT ALL. Then your PC in this "locked-down state" cannot be infected without you manually initiating an action that lets the malicious code in. That action might be insertion of media (like a USB drive), running a program, an auto-updater (which then happens to connect to a compromised server), or just browsing the web (malware getting in through "bugs in the browser or its plugins").


    I suppose the way that I'd put this is that malware is not powered by magic, and hackers aren't magical either. On most news networks and what we hear of hacking, you'd assume that they are magic powered, yet to break in to a computer or to infect a computer requires some entry point (some vulnerability / way in). On a typical Home Network (with IPv4) and Internet Sharing, there's NAT (Network Address Translation) / your router is protecting you to a degree here [from the outside], and ontop of that you have Windows' built in Firewall [for protection from the inside -- PC's on your own Local Network].

    ^ Neither being used (a router, or a local firewall) guarantees that you're fully closed up, since alot of people run servers and punch holes in the process ... though it's possible for you to configure the machine and your network in this way (with no inbound connections allowed). Another way to look at this is that your machine and home network on typical "default" configuration are akin to a bunker. Of course it's inconvenient to be trapped in a bunker with no access to the outside, and it's often desired to open that door. eg, allowing your buddies to connect to a file-server (especially over the Internet), or to launch a game-server.

    -----So, we start off fairly-safe with modern OS's [limited services running, and or bundled with a Firewall] ... yet soon enough as people install more and more software, that protection is no longer foolproof (as we've opened ourselves up). The key to safety is always opening up as little as possible.


    Right, if the folders are read-only to your gaming rig, then there's no way that your gaming PC can edit, create, or delete files there. The files would be completely safe from tamper.

    --You're talking about having your machines on your network configured such that each PC can see the NAS, and at the same time they cannot communicate to eachother? eg, having the ability to choose who each computer can talk with.


    Example:
    -Gaming PC can talk to NAS "ONLY".
    -Workstation PC can talk to NAS & router.

    ^ Most "managed" switches can do this, it's a feature called per-port VLAN. You'll definitely want to skim through their manual, sometimes they call it different things (per marketing)
    --The last that I've checked, the Netgear 8Port / GS108 was the cheapest model I'd seen that had this ability.

    Only to move data sadly.

    There may be other options there for viewing Anime such as KODI, which can stream files off of pretty much any type of file server. Thanks to access control / restrictions though, mounting your anime read-only should be a non risk. Your uploads folder (for game-videos) you could opt to leave unmounted and to do those with Filezilla if you chose to. --Always possible to use a mix of both mounting and non mounted transfer.

    Yeah, lockdown that bunker. :D
    --Ultimate way to keep data safe against malware and protect against ransomware, etc, is to not allow your files to be written to in the first place. To keep everything read-only unless writing it is actually needed.


    Key thing to remember is that Virus Scanners can only find "known malware". Zero-day / new threats, and simply "under the radar" or rare-threats (that there's no information on), won't be found until they're reported and studied. The more AV software that you run, the better the odds that you're going to catch anything simply due to more eyes, more virus definitions, more people involved in investigating malware.

    Ontop of that: Sandboxie is a very good idea to secure your media-player, as Sandboxie doesn't rely on heuristics or detection / rather it relies on "isolation" or containment (not letting the media player permanently write changes to your disks). Updating your players is also a good idea since the only way for 'code' to be executed from a media-file (remember, these aren't executables) is via a vulnerability (software bug) inside the player itself. [eg, macro language like Windows Media Player and opening web-links]


    If you backup your games on completion (not just savegames), then yes that would involve executable files and definitely there's more risk there. On the otherhand, I assume that you won't be using these games on anything other than the gaming PC, and you might be able to also "check" the game's files if a distribution network [such as Steam, uPlay, Origin, etc], was used in attaining them. The hash / integrity checking of any of those download networks would actually "find and purge" an infection if it got inside of the game's files. [via re-downloading any mismatch]

    --Maybe just make it a habit after restoring a game from backup (from the NAS) to run a verify first (where it's possible to).

    The more you can tolerate using Sandboxie, the better honestly. Pretty much anything can be run and even installed directly under it, though there may be some nuisance involved in updating software run from a sandbox. There's exceptions with some games due to anti-hack / cheat-protection (and in some cases just anti-debuggers -- rarely), though most games can be installed and run from a sandbox. I suggest not doing this for anything with online play, as it risks getting banned / red-flagging cheat-detection even if it works.


    --More or less I'd say you should use Sandboxie on your web-browser, email client, and Microsoft Office (due to Microsoft's record with Macro-Languages in Word and Excel) at the least. Doing this makes you much safer in that if you're infected browsing the Internet, and if your browser is isolated .. the infection is stopped there, can't spread, and is eliminated when the browser gets closed [sandbox wiped].

    You can set allowed locations for downloading files and also manually recover them [without having set rules] with Sandboxie. Sandboxie is a form of a HIPS (Host Intrusion Prevention System) in that "you" get to dictate where programs can and cannot modify your data.

    EDIT:
    --What sets Sandboxie apart from most HIPS software is that Sandboxie doesn't just have the ability to block access, it also can "simulate full-access" in a confined space. Full-access is simulated for files with an Overlay FileSystem (similar to OverlayFS in the Unix world), and Sandboxie does this for everything else too (memory mapped files, pipes, even administrative rights in confinement).

    That's one way to do it! Though as you get the hang of access rights, you'll probably do this by specific folders down the road. (or by both) Probably you'll be most concerned with if a PC is writing to some folder without your knowledge, since almost always this implies something afoul / malicious behavior.

    Right, I created a file and copied a file just to have "stuff" in a folder that I could generate hashes for (for the example).

    You'll find that I'm actually a noob when it comes to recording videos and doing any splicing or editing work. Yet I also haven't ever had a reason to record or edit much yet either... I think alot of it is just finding the incentive / purpose to learn what you need to really. For me, myself, it took losing ALOT of code before I started taking backups to the anal level. (RAID / NAS, external backups, online backups {on remote servers}, burned CD-backups, etc)

    Alot of this is actually what triggered my learning more and more Linux, I'm still primarily a Windows developer. Though as a whole it wasn't "Windows 10" but rather long-long before that, even before the Windows 7 days it was that desire to build exotic RAID setups that I found I just couldn't do with Windows. But yeah, once you have a reason to look at other OS's [just as with Mac in the old-days for image-editing] you move further from Windows in that you start realizing Windows just isn't best at everything.


    Today I'm at a point where I have more Linux and BSD PC's than Windows ... though really I think it's more that I just have found the beauty of each OS for what they're good at (rather than favoritism). It's hard to beat Windows for gaming, it's hard to beat Linux & BSD for servers, media-PC's, email, browsing, general workstations, etc. Don't need that many "gaming" machines, yet games are just one aspect of their use.


    --Oh yes, it's easier. HAHA! It may not be apparent, but ask just about anyone configuring a VPS or a dedicated-server in a datacenter what they'd like (for just about any server role -- {Windows, or Linux?}). Pretty close to 100% of people in the web-hosting field are going to ask for "Linux", not just because it's cheaper or that the install will use less-space, but because they'll finish faster. Install-faster, update-faster, configure-faster.

    You can install and configure Linux with Apache (for example) in literally "minutes", start to end of installation, and only occupying a few hundred megabytes of disk-space (for the entire OS). Meanwhile a Windows server with IIS? Heh, the Windows install alone [including updates] is going to take quite awhile. You'd be done setting up that Linux server before Windows even is installed and updated. ;)


    No problem, and sorry for the delay too. Things have been hectic here. Hopefully you've managed to find more originals on some of those images.
     
    Last edited: Apr 23, 2018
  6. 321Boom

    321Boom Member Guru

    Messages:
    111
    Likes Received:
    10
    GPU:
    GTX980 Ti
    Got it, thanks for the more detailed explanation, makes more sense now seeing that 0 could be a value of NULL, or a character representation.

    'as long as you're downloading using regular HTTP', does this mean there are other ways to download which are not just using regular HTTP (and will be more efficient/safer)? I'm keeping a lookout for the hashes on the websites as you stated, but these aren't available for all the data I download unfortunately, so it's not a foolproof method :/

    Can I use FTP to do my downloading? Is it safer/less chance of download corruption than regular HTTP?

    This sounds interesting, so by saying Wireshark is a packet capture/analysis tool, does that mean it will be handling the transfer of downloaded data on top of TCP? So it's a safer form/check of TCP?

    By analysis tool, is this something that regular TCP does too, or a feature implemented by Wireshark which makes it better than just replying only on TCP?

    Thanks, that cleared up some of the confusion knowing which programs go onto which OS (Windows or FreeNAS).

    If I set the gaming rig to 'no inbound connections allowed' would that mean I wouldn't be able to move any data from the server to the gaming rig (example games from the server to the SSD)? I understand that it's best to open up as little as possible, but there need to be a few things open to be able to use the PCs efficiently :/

    What do you mean 'your machine and home network on typical "default" configuration are akin to a bunker. Of course it's inconvenient to be trapped in a bunker with no access to the outside'. Does that mean I'm already set with 'no inbound connections' since I'm using the default configuration? That wouldn't make sense though because I do have access to the outside, I can connect to the internet, browse sites, etc :/

    Thanks for the confirmation and reassurance once again :)

    Yes very similar to what you stated 'having your machines on your network configured such that each PC can see the NAS, and at the same time they cannot communicate to eachother'. I need both the gaming rig, and the ECC desktop to see the NAS, but I definitely don't want the ECC desktop to see the gaming rig (or worse, the other way around, especially since the ECC desktop will have write-access to most of/all the NAS).

    The VLAN switches 'kind of' sound like what I'm aiming for, but not 100% sure they'll fulfill the purpose I have in mind, especially since I need the gaming rig to be completely without an internet connection almost all the time (it's no problem for me, I don't play online). What I'm thinking of is if I could have the gaming rig connected to the server in some way without requiring an internet connection to the gaming rig (which is why I suggested an intranet, or directly by UTP, don't know if these will work though). Remember, the gaming rig is with Windows 7, with Meltdown and Spectre patches disabled to avoid the performance hit, internet being toggled on and off with the Toggle-Internet .bat file we talked about earlier in the thread. I don't want the gaming rig having an internet connection to the outside (especially due to the Meltdown/Spectre patches disabled), but I would like it to have access to the NAS in some way (without having to Enable the Meltdown/Spectre patches again, restart the pc for the patches to take effect, then switch on the internet with the Toggle-Internet .bat file)(quite a tedious process having to do this every time when wanting to connect to the server, then disable everything and restart again when playing a game).

    In a more simplified way to look at it, I'd like the gaming rig to have no connection to the internet/outside world (it will only have an internet connection very rarely when needing to update drivers or something) but at the same time it could always 'see' the NAS as a mounted drive/external drive, so if I want to check out an old gaming recording that I stored on the NAS, or view some anime art I don't have to keep on enabling the patches again, restarting the gaming rig, etc.

    Got it, only for data transfer. Hmm that sounds like a good idea about using FileZilla to move the gaming recordings, that way there wouldn't even be a need for a folder with write-access from the gaming rig correct? Or would I still need this folder for FileZilla to have somewhere to copy the recordings to?

    I'll have to give it a try and see how it goes about mounting the anime and anime art as read-only since it could really make my life more difficult, especially when it comes to the anime art since I do move these around sometimes for better organizing. (I can't move folders if they're set to read-only right?)

    If a folder is set to read-only, can I add new files in there, or even that is prohibited (not just deleting and editing)? I have a habit of taking notes after watching an anime, and I put these notes in the corresponding folder of that anime, so it would be troublesome if I can't add these notes. (unless it's a simple process of switching from read-only to write-allowed for a couple of seconds till I put these notes in then switch back to read only).

    Since we're on the read-only subject, is the setting of which folders will be read-only something I will be setting from the NAS itself, or using the ECC desktop to set which folders are read-only (and setting for which PCs the read-only applies)?

    Also, if something is set to read-only, it wouldn't give me trouble/error messages when taking back ups with rsync and copying with TeraCopy to external drives?

    Sounds like more +1s towards Sandboxie then due to it's isolation. Seems like a very handy program to have which I will also start implementing. Thanks for suggesting it, this isn't the first time you've pointed it out, and I could see where it could be advantageous for many scenarios.

    Yep, talking about backing up the full game, since it will be English patched, and any other patches/updates installed to the game, makes it easier to just pick up and play again rather than having to install all that all over again. Yes only the gaming PC will be running the games, no need to ever execute them from the ECC desktop or NAS. Unfortunately I don't think I could check the hashes for the games, they're on cds, not a distribution network (remember they're old games). Unless there's a way to check a hash from a cd? But won't the hash differ if so since I would have added an English patch, and other update patches? (it won't be the exact same content as it was on the original cd, or am I confusing something here?).

    Awesome, even more +1s towards Sandboxie then. I'll definitely implement this, especially if it won't hinder the way I could save anime art and data (since it could be easily retrieved from the allowed locations where it could download). Makes perfect sense having an isolated session that gets wiped as soon as it's closed, so as you said, infection would be stopped there, and not get into my files. Thanks again for the suggestion.

    Thanks, this is very good advice. I think a full hash generation would still be more beneficial though to have a record of literally 100% of all the data that's stored on it (the integrity of ALL the data is important). I think doing it for specific folders is more for troubleshooting to determine from where the foul play is coming from correct?

    Yep, same here, usually there has to be something to spark that interest in order to start putting in the effort and learning. I've been playing games for 25 years, yet I felt the need to start recording 2 years ago when I picked up shmups (shmups are ridiculously difficult games, especially the Japanese bullet-hell ones, so it's nice having a recording of your achievement :)).

    Thankfully I've never lost any data in all these years, but I've always wanted a server (ease of access to your data, multiple mirrors due to RAID). Shame your experience wasn't similar :( I've heard in multiple instances that RAID setups are dealt with much better on Linux based systems, so you're not the only one suggesting this route. I understand and agree, it's not just favouritism, every OS has it's own strong suites and shortcomings, so you need to use each at what it does best :)

    Ahhh, you're a developer, no wonder most of this stuff comes naturally to you then and code doesn't scare you lol.

    Wow, I've read and heard that most people opt for Linux based system due to reliability and extra features, but didn't know it was easier as well! Two birds with one stone then choosing Linux for the NAS!

    Haha that's quite a difference having the Linux server set up by the time the Windows one is still installing! Interesting points to know. Thanks for the insight, as always.

    Sorry for the delay once again, as you said, it's really been hectic here as well. Finally found some time to sit down and reply since it's Saturday. I honestly didn't have the time to try the Search-by-Image yet, but I'm sure it will come in handy, even if it's not able to find all the images I need to bring up again, I'm sure it will be beneficial :) Thanks for all your help, and hope you have a nice relaxing weekend! :D
     
  7. A2Razor

    A2Razor Master Guru

    Messages:
    451
    Likes Received:
    42
    GPU:
    ASUS R9 Fury X
    --Sadly plain old FTP doesn't do anything for integrity checking.

    SFTP & SCP do provide integrity checking (via SSH), so those two are much safer than FTP or HTTP. HTTPS (secure) can provide decent integrity checking depending the cipher used. Though HTTPS is usually not mandatory on websites without explicit security requirements (where money is involved).

    Wireshark is for inspecting the communication of other software. It doesn't provide a protocol or do any transfers on its own, rather it lets you inspect what's happening under the hood.
    --Think of this like a debugger in the network-world.

    Outbound = connections initiated from the machine itself. (eg, acting as a client)
    Inbound = connections from the outside world in to the computer. (acting as a server)

    If a computer can only act as a client, and if there's no client software installed on the machine that doesn't "do stuff" without you giving an order ... then that machine would be untouchable & unable to be compromised unless a connection to the outside world is opened up from inside. Put another way, it could still be vulnerable, yet vulnerability would require human interaction. (left alone with no user the machine is "safe")

    Or rather, I assume that once you record videos you then drop them off on the NAS (in your storage pool), and thus there's no need for a local file-server run on the gaming-machine. When you go to edit these videos, then they're available on the NAS from your work-machine / workstation.


    The port-based VLAN on a managed switch can definitely do what you want here. That said, port-based VLAN would be an absolute hassle (nightmare) to change constantly / has no easy method to be changed on the fly in this case. [you'd have to re-flash the port configuration to the switch each time]

    --One solution to the problem of wanting to toggle Internet Access off "quickly" would be to just use two ports on the managed-switch, and assign one port as Internet Access "only", with another port as "NAS" only. You could buy a cheap 10$ NIC and shove a second network-card in the Gaming PC.

    --Another less complex method would be to delete and re-set the default gateway (as a toggle switch). Though this won't necessarily kill existing already-connected sessions, IMO.

    Here's an example what I mean by that:
    Code:
    Port #1: NAS
    Port #2: Router
    Port #3: Workstation PC
    Port #4: Gaming PC (to NAS)
    Port #5: Gaming PC (to Internet)
    
    Port 1 VLAN rules: [1, 3, 4]
    1: yes, 2: no, 3: yes, 4: yes, 5: no, 6: no, 7: no, 8: no
    
    Port 2 VLAN rules: [2, 3, 5]
    1: no, 2: yes, 3: yes, 4: no, 5: yes, 6: no, 7: no, 8: no
    
    Port 3 VLAN rules: [1, 2, 3]
    1: yes, 2: yes, 3: yes, 4: no, 5: no, 6: no, 7: no, 8: no
    
    Port 4 VLAN rules: [1, 4]
    1: yes, 2: no, 3: no, 4: yes, 5: no, 6: no, 7: no, 8: no
    
    Port 5 VLAN rules: [2, 5]
    1: no, 2: yes, 3: no, 4: no, 5: yes, 6: no, 7: no, 8: no
    ^ This assumes an 8 port managed switch (generally the smallest that they come) with per-port VLAN tagging. The {yes / no} for each port is whether or not each port is allowed to communicate with the others. So, think of this like bridging those ports together, only this time you have control over which can send data to the others.

    EDIT / NOTE: You would need to put the NAS on a second subnet. Having two NICs on the same subnet is a no-no for Windows.

    So, you could have say:
    Code:
    192.168.0.1 -- {NAS}
    192.168.0.4 -- Gaming PC (NAS Access)
    
    192.168.1.1 -- {NAS}
    192.168.1.2 -- Router
    192.168.1.3 -- Workstation
    192.168.1.4 -- Gaming PC (Internet Access)
    -The NAS is a single NIC with two IP's, one on each subnet. The "Gaming PC" entries are one IP per NIC / port.

    Still need a folder with write-access for the gaming rig (to drop your videos in), yet the difference is that you won't be mounting this folder. Since you're not mounting the folder, there's no visibility (like there would be as a drive) to other software on the PC directly. This is the whole idea of security through obscurity in that it'd be much harder for any malicious code to write to the folder(s), as they would have to be programmed to be aware of servers entered in say "Filezilla".

    -- You (as the user) would of course know how to access that account & folder on the NAS to write your videos [with a tool such as Filezilla], yet malware on the machine probably has no knowledge of this. (which makes it somewhat safer if that makes sense)

    --Right, if you wanted to move around files you'd need to do it from the NAS (over an SSH terminal). Assuming that no machine has the permissions to do this graphically (drag & drop or cut & paste) then this would be a pain in the butt.

    You could change permissions pretty quick from the NAS / web-administration, but definitely not something you can do instantly or every few seconds (without that getting annoying fast). File "creation" can be allowed without deletion or modification, though sadly there's probably going to be a need to edit notes and go back / re-save already existing files.

    Different folders can have different write permissions though, so you might be able to still have different restrictions if you could live with a notes-folder (read-write) and video-folder (read-only).

    In the case of FreeNAS you'll probably do it from their accounts "web-GUI". The NAS itself won't actually have a desktop-environment (none installed / just a text-terminal), and the settings are intended to be done from a web-browser on another system [for the non-advanced stuff].

    --So I'd say: Remotely from your workstation over a web-browser.

    Nope. Backups from a read-only volume pose no problem to any tool of your choice. Copies from them with Windows Explorer, TeraCopy, xcopy, robocopy, rsync, etc, etc, will work just fine as long as you have write permissions to where you're writing. Nothing gets written in to the source-folder.

    Yeah, Sandboxie is truly awesome. It's just about the most secure sandbox isolation product on the market for Windows, worth every penny for a license, free for browser-isolation even without a license (single sandbox).

    --It may not be a full blown VirtualMachine like VMWare or VirtualBox, yet Sandboxie can isolate pretty much every known virus there is.

    Bear in mind that Sandboxie by default won't stop malware inside a Sandbox from reading data outside the Sandbox, yet the default behavior can stop malware from spreading. With manual configuration, it can also block reading folders. [eg, you can manually set folders that read-access is disallowed for] Expect some nuisances, and expect it to take some time to get used to and understand the whole isolation concept, though really really worth it.

    ^ The big VM softwares (VirtualBox and VMWare) block everything (since a guest OS can't read the disks of the host without special setup), and technically they're even more secure. You can even integrate a Linux Guest VM with a Windows Host, having Firefox run under the VM -- rendered on the host -- using something like VcXsrv, combined with PuTTy or Tunnelier (Bitvise client). Though this regulates you to Linux and is pretty inconvenient.

    Yeah, definitely both can be useful. Not always just tamper detection, although obviously great for this too. Those hash dumps of a folder are like a "snapshot in time" without taking up the space of copying everything and good when you want to see changes (deletions, creation, modification of files) after running any software-update. You'll probably find you use them not just for probing for tamper and corruption.

    --Though seriously nothing gives more peace of mind (per a sanity check) after suspecting infection to run a hashcheck of the Windows directory, user directory, etc, and find that absolutely nothing nothing has changed.


    Nah, Java, C#, Python, and other high-level languages absolutely terrify me (since they're gaining tracking in the business world). I like having full control, what can I say. :D

    Yeah, best example I can give here is that I restored an image of a Linux box from mid 2017. ~700 updates to install! Done installing in under 30 minutes, no SSD, lol. Imagine that type of update on Windows when an OS has a full year of updates to go through.... The difference of speed of updating compared to Windows has gotten absolutely insane. In Windows' defense, the Linux package systems aren't done transactional with rollback support like Windows is. On Linux distros, going back to old versions or rolling back packages usually means re-acquiring the old versions from their respective repositories.
     
    Last edited: May 12, 2018

Share This Page