Backup Strategy revised...
Monday, April 13, 2009 at 5:20PM So my external storage comprises 2 x RAID arrays both configured as RAID 0 or scary RAID as some people like to call it.
The reason I originally went with RAID 0 was two fold, one it was fast and secondly, my eSata card didn't support RAID 5. The reason for two RAIDs was to use one as a master and one as a backup.
Sweet!
Until the backup RAID failed for no apparent reason a couple of days ago. Ok, so I still had the main RAID but as this was also in RAID 0 format, it gave me a couple of hours of palpitations whilst I made some space available on the Mac Pros to make an immediate copy. Unless you've got two copies of something, you ain't got a copy at all!
Actually, it wasn't that bad as I'd taken a full backup of the RAID up to the end of February and moved that offsite. So I really did have two copies of the bulk of my project files plus most if not all of my really important documents are backed up between Amazon S3 and Dropbox. Plus, my video project files since February were also stored on one of the Mac Pros, so it wasn't too perilous.
But I had to fix the backup RAID and decided that I needed to come up with an alternative to RAID 0 moving forward, that included the provision for moving backups off site. BTW, I can't really rely on a cloud based backup solution for my video projects, the shear amount of data generated each week makes it impossible with my limited upload speed. I have to backup to physical media.
So this is what I'm planning....
The SCO Archive (all my screencast video project files) I've pruned down to 1.1TB. I had a lot of intermediate files that I could delete and recreate if needed so there was no point keeping those. As it's over 1TB, I'll need to split it over separate drives (didn't really want to buy any more drives and the biggest I have are 1TB) so I'll probably go for a Firewire 800 based Drobo as permanent storage. I have 4x750GB in one of the existing RAIDS that will give me 2TB space so will give me another years expansion even without upgrading the drives. I know the new Drobo Pro is out soon, but I'll wait until they are tried and tested and also a lot cheaper!
I'll also be creating a mirrored pair of 1TB drives for new additions to the SCO video archive. So when I create new content, this will be backed up to the mirrored pair and then also to the Drobo. The beauty of a mirrored pair (or RAID 1) is that each disk is an exact copy of each other. If one disk fails, your data is still on the other disk and when you introduce a new drive, the mirror is created.
There is one other benefit too! At monthly intervals, I can deliberately break the mirror and convert one of the drives into a standard drive which then becomes my offsite backup. Whack in a new drive and re-create the mirror and repeat the cycle for as many times as I want.
As well as the SCO Archive mirror, I'll create a 1TB mirror for all my media, again, will probably last for a while before I fill it up. Not too concerned about off site backups for this but would be a nice to have!
And finally, another mirrored pair for any other data that I need to keep, not connected with video projects. I may also mirror this to the cloud using Jungledisk, Mozy or BackBlaze. I may wait to see if Amazon introduce a service that allows you to send in a drive for them to copy locally in to S3 which you can then incrementally update - they are thinking about it and it would be excellent if they did introduce it.
In the medium term, I can well see me investing in a Drobo Pro, once the FW Drobo starts to reach capacity with the SCO Archive, but I'll still need the mirrored pairs for offsite backup.
I'd probably re-use the FW Drobo as a separate media only storage device, perhaps hooked up to the Mac mini once in place as the Media Center (or Apple TV Take 3 whenever that appears!)
Anything I've left out?
Reader Comments (17)
I have a Drobo which is covered for one drive failure at a time but what about electronics failure?
Scott Bourne has said that they took all the drives out of one Drobo and inserted them in another without problems but!!!
My Drobo is in a DroboShare so that it can be moved to separate building or garage or even a shed at the bottom of the garden!
Just buy a Drobo... I have both the Drobo, and have just bought a Drobo Pro....
One has 8TB of storage in it, and the Drobo Pro has 16TB in it.... I have filled up the Drobo, with Dicom Sets.... and am considering getting an xserve to connect them both up to!!!
I would say that it is fast enough that in Raid 1... and easily has a large enough space :D
I would however... have it setup so it is mirrored... so well... It just gives you a piece of mind doesn't it!!!
I love the Drobo, but I'm scared by the idea of electronics failure...Your data are inaccessible till you find a replacement unit, if they are still available at the time of failure...
All things considered, I have settled for software RAID: I had an issue with my drive this very weekend, and the model I used was out of stock. In my case all is good, I purchased a different model and my data are readily available, even the RAID structure is still perfectly consistent. So, no Drobo or other hardware RAID solutions for me, unless I can afford to buy the corresponding maintenance service from the producer.
DON - I was smitten by the Drobo and bought one, but returned it. Several reasons:
1) It is SLOW SLOW SLOW (even using FW800!).
2) I found I couldn't trust the software... when I clicked on Check for Updates in the Drobo utility it said I had the latest version installed when I didn't.
3) You can't remove a drive from a Drobo and get data off it, except by inserting it in another Drobo. So you need a Drobo to back up your Drobo.
4) The Drobo user Forums (www.drobospace.com) are full of horror stories and complaints. I found out about the Forums AFTER I purchased it. And the Forums are PASSWORD PROTECTED so that people like you can't casually browse them as you're conisdering a purchase.
5) I wanted the Drobo to be my boot volume, but Data Robotics tech support told me after I purchased the Drobo they not only do not recommend such a configuration but they also will not support it. Nowhere on their website is this explained, especially for pre-sale info. I asked several times why they don't support this configuration and the most information I could find out is that it's because of their proprietary driver.
6) I purchased the Drobo+2TB bundle. You'd expect the drives to be 7200RPM, right? So did the dealer. Turns out they were only 5400RPM so the dealer exchanged the Drobo bundle for a Drobo without drives. Take a good look at the specs for the Drobo (with the two included drives). See if you can figure out the speed of the drives (there's no speed listed).
Hope this is helpful info. Please keep us informed of your decision.
Wow, this whole Drobo thing seems to be a very emotive issue!
Just so we're all on the straight and narrow, I've no links to Drobo as a company, so my plans on using one is purely based on published functionality. I suppose I could have tried to get a discount or even a "trial" one via Scott Bourne, but I decided not to go down that route. So I'm completely independent from Drobo.
Right!
The decison to go down the Drobo route was based on a few factors:
1 - I just wanted a simple solution to maintain a growing pool of online data. The Drobo would not be the only backup of the SCO Archive, it's intended as a convenient online resource for the SCO Archive files. I could have just backed up to raw drives and kept them in a cupboard, but I like having them online.
2 - The Drobo is only intended as backup, I'll not be accessing the drive directly, it will just be the recipient of overnight backups so speed is really not that critical.
3 - It's not the only backup. The archive data is held on an offsite backup, any new data is transferred to an intermediate mirrored pair of drives which is also used to create additional offsite backups.
4 - It's a relatively cheap solution as I already have the drives to use in the Drobo. I deliberately didn't go with the Pro as that's still a bit expensive!
I'll certainly let you know how I get on once I've had a play with it!
Just for a bit of balance, I'd like to respond to Dave's points in comments #4
1) Of course it is slow - it is constantly shuffling data around to give the best balance of space and redundancy. That is what it is SUPPOSED to do. It has NEVER been marketed as a performance device
2) The software is a simple admin reporting tool. It is not required to run the drive, so having the absolute latest update is not really that critical. I've certainly never had a problem with the self-update feature, but when accessing stuff over the Internet your mileage may always vary. As it is not crucial, seems a bit unfair to castigate the entire product about this
3) Of course you can't unplug a drive and read it directly - the data is striped across all of the drives. You can't do that with a £5k SAN system either, or anything that uses data redundancy across multiple devices.
4) The Drobo forums are password protected, true - and like most Internet forums require a free registration to access to avoid spammers. There is certainly no need to purchase a Drobo to get in there. And it should be of no surprise that a support forum is full of people with issues - that is what support forums exist for!
5) Drobo is neither designed nor marketed on disk performance - it's a storage device, not an uber-wicked drive array. So booting off it will do nothing but slow down your computer. Surely that is why it is not supported?
6) See point 5. The disk spindle speed is not a performance bottleneck with a Drobo, it is the overhead of maintaining the data redundancy. So having faster drives is wasted money - and the last time I looked, their bundle drive prices were pretty cheap compared to sourcing directly.
There seems to be a real fanboy backlash against Drobo going on at the moment. I don't know if it is because the product messaging is wrong, or whether people have issues with the way it is being promoted (I know that Leo and Scott have their detractors as well, and those detractors are often have black and white opinions of everything they do). But there is nothing else on the market that can do what it can do, and I think Don's experience with RAID config and failure typifies what can happen if you get that wrong and the worst happens.
Why is everyone so afraid of getting into raid .. its not that hard (ok you might need to read a manual)
but once it's setup you can "forget" about it.
personally I looked at the cost of drobo's and qnaps and for the same cost as the bare unit i bought a pc parts installed solaris and build a 6 drive mirror set with zfs - exported with iscsi
basically for less then the drobo i have the same as a drobo pro
I was thinking of following Don's footsteps and screencasting the software setup of this .. anyone interested?
I think a lot of people are missunderstanding what the DroboPro (and indeed the Drobo) is and what it's designed for. As a systems engineer with many years of experience working with all sorts of storage solutions, I find the Drobo a new and fresh approach to "RAID" style storage and at a fraction of the cost of a anything similar.
RAID stands for Redundant Array of Inexpensive Disks. When RAID was first introduced (Yes I do remember when that was) disk arrays were only available on Mid-Range computers such as the IBM AS/400, the drives were DASD and very expensive. With the intrduction of SCSI drives, RAID was possible and considered cheap beacause you didn't need expensive DASD drives. However, SCSI wasn't that cheap either.
It's only in the last couple of years with the introduction of SATA drives has RAID trully meant inexpensive. So inexpensive that many mothervoards now support RAID and have SATA RAID controllers built in. Not that I recommend using them, I'll write a blog post as to why.
There is no RAID 5 solution (which the Drobo is very similar to) that allows you to remove one drive and read data from it. The whole idea of RAID 5 is redundancy, 1 disk failure in the array means that your data is protected. I'm being very simplistic here by the way.
There are tools around that let you do a partial recovery of data from a disjointed RAID. I have successfully done this. By the way the Drobo uses a standard format which you choose i.r. NTFS, HFS, etc. when you setup the Drobo so there is no propriatory formatting.
The complaints that I have seen regarding the Drobo seem to be from people who do not understand the concept. Those of us who do think these devices are just amazing. There is still room for improvement and I have no doubt that Data Robotics are committed to their product and we will see software updates and new models in the future.
Thanks for the update Paul.
I think you are correct in that it seems the basic concept of the Drobo (and it's main benefit) are being overlooked in that it's not a typical RAID solution. RAID 5 is great but you need a special RAID card to support it (it's not supported in software by OSX) but even RAID 5 doesn't allow you add storage on the fly and have it automatically re-configured into the data pool.
I can't think of another storage product that allows you to do this seamlessly at such low cost.
As an aside, I've just seen the FW800 Drobo on special offer over at eBuyer.com with a free DroboShare device worth £169 for free. http://www.ebuyer.com/product/161054
That deal seems good enough for me to take the plunge!
I have a FW800 Drobo - for use as a secondary disk and iTunes library and I have nothing but praise for them and their product. A few points first:
1. It's not fast - but it's not horrendously slow either. I can use it for audio/HD video - and that suits me fine.
2. Only if you do disk-mirroring can you pull a drive out of one RAID unit and put it into another. Any kind of RAID-5 and you are tied to the manufacturer of the unit. You can't put a RAID-5 disk from one manufacturer into a unit of another ... (so no difference from Drobo here).
I had a problem with my FW800 unit. It would occasionally cut out under high load. I went through some diagnostics steps with their US-based support team, sent them some logs. They then deduced I needed another unit. They shipped me a replacement unit, I swapped the disks from the old unit to the new, sent the old one back and job done. They were very professional and responsive throughout.
I'm very happy with both them and their product.
DON - One thing I forgot to mention about the Drobo Forum. It not only requires a login, but you must own a Drobo and provide a Drobo SERIAL NUMBER to access the forum! This is a shoddy business practice that prevents people from researching the product fully. Why would they have a closed forum? Caveat emptor!
@David Cohen - David, you *DO* need to purchase a Drobo to access the Forums. After you register a username and password you receive an email with a link to verify your email address. After you click on the link you must then enter your Drobo Serial Number. The words are: "Customer Login - In order to access customer-only areas of the site, please enter a serial number for your Drobo or DroboShare below." Sorry, David, this rubs me wrong... people should be able to read about the problems users are having BEFORE they purchase.
Dave, you are correct, point taken!
That is a new policy - I browsed the forums extensively before I got my Drobo, so I never registered with a serial number. I concur that is disappointing, Dave - but I stand by my other comments.
I agree, closed product forums are a pain and I dislike them also. Whilst being pretty rare occurrence in the general consumer market place, which the Drobo seems to be targeted at, it is unfortunatelly all too common in the enterprise world.
I had to purchase, register, re-register and register each plug-in's serial number after purchasing a SonicWall firewall only to discover that I wasn't the only one who thought the product was a heap of junk.
I'm not saying this is the case with Drobo, I am certainly more than happy with my 1st gen Drobo but it is, in my opinion, inherantly bad practice to have a closed user forum.
So, Don, did you get the Drobo yet? Are we going to see a show on the Drobo? If so, please include the Droboshare. Since you have experience with RAID setups, please evaluate the performance and pros & cons.
Drobo and DroboShare due in for next week. Give me a few weeks to have a play with it.