Photography & Technology Part 2.5; Data Storage
For me, this was the most difficult part to write for this article series. There are so many options and in the last few months, the choices were tough. Until now.
It’s not like a computer or monitor where you can play with it and decide whether you like the devices or not. You pretty much have to rely on specifications. And that’s not foolproof.
I bought a Western Digital ‘Green’ hard drive once. Big mistake. I love Western Digital hard drives. Never a problem with the brand. But these drives have the platters spin down a short time they are last used. Which means if you want to access data, you have to wait for the platters to spin back up again. This adds a huge amount of time to opening data files! It took me too long to figure that out and could ‘t return the drive. I couldn’t wait to upgrade!
Starting out and now, I use a simple USB drive. Simply: It works. In the beginning, USB 2.0 sufficed more or less. While opening and saving Photoshop files took a while, it was still a short time when you look at the whole process. Exporting many event files wasn’t really an issue as my computer’s processor was the bottleneck.
When I upgraded to my Mac mini and it came with USB 3.0, I upgraded the external drive with USB 3.0. Data processing speed improved a good deal. The performance increase was nice especially when working in Photoshop. At the time, my Photoshop files were averaging 1GB – 2GB in size. Mostly because my ‘eye’ wasn’t as developed as it is now and I wouldn’t notice errors in the image during the photo shoot. Then Photoshop files were big because of the problems I had to fix. Then, files took about 20 – 30 seconds to open. Not such a big deal when opening and closing a file. But when you want to save the file while working, it’s a drag having to wait. But as in many technologies: performance costs.
I think I’m making this sound like the end of the world. For what it is, the single USB 3.0 hard drive is a really great bang for the buck. I get a little nostalgic when I think of my single USB drive system. It’s done right by me for years. Never a problem. Well how about if it fails you ask? Online backup. How about failure in general? I hear people complain about drive failures. I could count the number of drive failures I’ve had with one hand. Take into consideration I’ve installed thousands of computer systems. Yes, it happens if you buy junk. Or you have your toaster connected in the same power circuit your computer is on.
While simply purchasing a bigger hard drive and transferring data is a simple process, it’s not a great long-term solution that’s going to take me to the next level. Performance isn’t going to improve and there is no fault-tolerance. Tempted to buy a single drive since I saw an 8TB version advertised a couple of days ago! Well, not really since it cost $599!
The next logical choice is a RAID-based system. For those who don’t know, basically a device that groups several hard drives together to act as one. The advantage is capacity, speed and redundancy. If you plan it right, you can also upgrade hard drives at your convenience to increase capacity. Okay, I know what I need. So what’s the problem?
Ignorance is Bliss
I researched systems which were going to get me the biggest bang for the buck. Because of my technology background, I got stuck in a loop where I couldn’t decide which system to go with. The pros and cons of each system were complete opposites with the other system. That sometimes happens when you know too much! Here are the systems I was looking at:
Direct Attached Storage (DAS). This RAID system is connected to a computer directly via USB or Thunderbolt. The great advantage to this system is cost. The RAID enclosure is relatively cheap. But the big problem I have with it: Buy-in cost.
To make this type of system a truly long-term solution, I would need to purchase SSD systems which are at least 2x as expensive as regular hard drives. If I use regular hard drives, the performance advantage won’t be as long-term. I’d have to upgrade in a couple of years and this would increase the long-term cost of a solution.
The Other World Computing DAS enclosure pictured is definitely on my short list. It’s a tiny box that has hard drive connectors and very little intelligence. The computer’s operating system controls the DAS. With some decent-sized SSDs installed, this unit would scream performance. But cost would is a nudge prohibitive. I could wait a year for the SSDs I want to come down in price.
Network Attached Storage (NAS). This was my first choice especially considering the experience I have with them. It would simply be connected to the network and would be accessible from any device within or outside the network. Plus all the systems I was looking at have extra application software installed turning them into mini-server systems. I can run all kinds of software applications on them.
I’ve used a NAS plenty of times in my Information Technology profession. Probably in the last 7 years, they’ve made so much progress. In the beginning, they were simply dumb boxes with just enough software for management. Then they just kept growing in features and more self-contained. When it came to replace a client’s server, I installed a NAS instead. Superior in every way to a traditional computer server. I have clients using the 5-bay version of the Synology pictured here. The Synology units are simply bullet-proof.
But there was a big problem with a NAS solution: Network speed.
Currently my internal network is limited to a standard 1Gb/second. Which isn’t much faster if at all than USB 3. So no matter how fast the NAS is, I won’t see any speed improvement from a standard, single USB 3.0 drive. I could upgrade to a 10Gb/second networking system. But that solution is hugely expensive and completely over-kill especially when cost is considered.
So I thought I would go with the NAS. While the network speed isn’t very good, it’s a long-term solution and would suffice until I get a DAS system to complement it. The NAS was always going to be an archive system and not for day-to-day usage anyway. I would at least have capacity and data-redundancy.
I had made my decision and then it was to set a goal to purchase the unit. It was not going to be a perfect solution. Just about every type of gadget and technology is always waiting for it. At some point, you simply take the plunge. But then a dream came to realization.
The QNAP TVS-871T NAS/DAS. This system has the advantage of both RAID worlds: DAS and NAS connections! Which means I can have the device attached to both the computer directly and on the network at the same time! Thunderbolt 2 and network connections are built-in! QNAP will also have a 10Gb network connection for the unit that will give it an incredibly long and useful life. Though I think that option will be an add-on board. I wouldn’t use that option for about three years and then the price will drop. Perfect.
This system will come in a 4-drive or 8-drive enclosure. I’ll probably go with the 8-drive system as the price difference shouldn’t be that much more and provide increased capacity and performance. Even with regular hard drives, performance will be nothing but fantastic and will make a great DAS. Really, I won’t have to upgrade the enclosure for at least 3 years and most likely five. If at all. I won’t need a DAS anytime soon if the speed is where I think it will be. An 8-bay DAS has incredible performance regardless whether standard hard drives or SSDs are installed.
Remember I talked about software being installed in NAS systems? Well one of those pieces of software is called Virtualization. This software mimics how a typical PC desktop computer would work. What may not be apparent is that you can install software like Microsoft Windows or Linux on the NAS systems! Yes, you can run a real computer in the NAS! Think about it: running multiple computers to process dozens of images at the same time. If the virtual computer crashes, it just crashes the tiny software process running in the NAS. The NAS isn’t affected at all. Just restart the process. I’m not so busy that I need to do this. Yet. But it adds to the long term value.
If price isn’t prohibitive, this system will be without compromises.
So now you know why I created a special episode on storage! Probably the biggest and most important issue and headache that photographers face with their computer systems. And it’s going to be even more important with the average megapixels in camera sensors increasing every year. But the good thing in my case is that my ‘eye’ is much better developed and I catch problems during the photo shoot. Which leads me to less post-editing and smaller file sizes. Thank goodness!
Next and final article on the series: My Future Computer System