Home » Articles posted by gbr (Page 5)

Drobo Speed vs Linux RAID5 Speed

I recently had a failure on my 4 TB RAID5 array.  It happened when I moved the server from the rack to my office, in preparation for converting it to a desktop system.  It would still host the RAID array, but it would no longer be running all the virtual machines.

During the move, I also upgraded the RAM.  Shortly after powering the server on, I noticed that any access to my harddrives (RAID or not) was extremely slow.  The kernel showed high load, with 98% being in system.  Wow.  Before I had time to realize it was the RAM I added, I lost a drive on the RAID array.  I pulled the RAM, and things went back to normal.  I added a new drive and started rebuilding the array.  At 97.1% complete a second drive got a permanent read error.  Good bye array.

I managed to get the array back up degraded, but I couldn’t rebuild it.  Now I know I could probably have used dd to copy the disk to a new one, get another new drive, and rebuild that way.  Oh well, hindsight is 20/20.

I needed to get my data off the degraded array.  There was no critical data on it, so it had never been backed up.  How am I going to back up 2+ TB of data anyway.  It mostly contained over 1000 recordings from MythTV, some audio and video, and pictures.  Still, if I could save the data, I really wanted to.  So, I went out and purchased a Drobo.  Drobo’s don’t really like Linux that much, so I hooked it up to my Mac Mini and started copying data over via rsync.  Wow, was that slow!

It turns out the Drobo is a really. slow. device.  Although it can handle watching live TV via MythTV, it barely keeps up.  Anything else done on the Drobo kills the TV.  Granted, the Drobo is a Firewire 800 device, but it still slower than that would imply.  Knowing that I will be recording and watching 2 HD streams at the same time, I knew the Drobo wouldn’t cut it for me.  For example, a Drobo rebuild could take days, and it’s all internal to the Drobo, no firewire used.  But, during rebuild, live TV was impossible to watch.

I built a new 4 disk 2.8 TB RAID5 array under Linux, and started transferring live TV and recording to the RAID5 array.  Things got much better  I decided to do some tests.

The Drobo, connected via Firewire 800, writing a 16GB file took 628.89 seconds (10.48 minutes) ~25 MB/s
The Linux RAID 5, writing a 16GB file took 135.59 seconds (2.26 minutes) ~121 MB/s

25 MB/s is doable for two HD channels, except for the issue that nothing else could use the drives at the same time.

I then tested the connection via the network.  My MythTV Server is a simple Pentium M 1.5 GHz system, and it uses remote Samba shares for it’s storage, over GB ethernet.

The Drobo via OSX SMB share, same 16 GB file took 1451.37 seconds (24.19 minutes) ~ 11.3 MB/s
The Linux RAID5 via Linux SMB share, same 16 GB file took 459.08 seconds (7.65 minutes) ~ 35.7 MB/s

Ummmm.  Ouch!  The Mac Mini is a Core2Duo at 2.6 GHz, the Linux is a Core2Quad at 2.6 GHz.  Neither used any noticeable CPU during the file transfer.

Tweaking the OSX smb.conf file didn’t help any.  What did help was specified here.  That alone changed my speed to 16 MB/s.  Still not good enough, but better.

The Drobo is a cool device, and has it’s place.  But not in  a media streaming type environment.  I’ll use the Drobo to store my audio, pictures, and movies, but not my live TV recordings for MythTV.

Wordless Wednesday

What happened to Wordless Wednesday

The blog was being filled with Wordless Wednesdays, and no actual content.

We’ll see if we can remedy that.  Wordless Wednesday is still around, just relegated to fewer time slots.

Wordless Wednesday

Wordless Wednesday

Wordless Wednesday

cleaner

Wordless Wednesday

caroptical

A good friend of mine, Rob Peters, posted in his blog some pearls of wisdom he gleaned from Jennifer Kornelsen, Ph.D. :

1)  The simplest explanation is the best. (i.e. the most likely, the most accurate, the most truthful)

2)  The data is what it is. (trust it, let it be…)

3)  If you’re nervous and think you’re going to puke, eat something colourful! (at least then it will be Spectacular!)

Rob found these pearls to be as true in Design as in Neurophysiology.  I’m here to say they also apply extremely well to Software Design/Development, and IT work.  It’s unfortunate that, in software and IT at least, a lot of people forget item number one.

However, in the writing of fiction, I’m not so sure (well, except for item 3), at least not on the surface.  For item 1, when the reader reaches the end of a book, the final explanation of event should be clearly evident and obvious, and yes, even simple.  But during the reading of the book, the simplest explanation of the events occurring is usually the one you want the reader to follow, but should not be the true reason.  You gotta keep ‘em interested.

In writing fiction, item 2 closely correlates to item 1.  This can be especially true when reading a first person narrative, where everything presented to the reader is viewed through the eyes of a single (or potentially multiple first person) characters.  If the character looks at the world through rose colored glasses (cliché), then that is how the reader will interpret the events (data) in the book, and therefor, the data is tainted.

Perhaps I should re-phrase my original statement.  To the reader of fiction, the above pearls should not be true, but to the writer, they probably should.

Wordless Wednesday

snowcar

"Climbers" almost die

I hate it when the ‘press’ calls anybody on the side of a hill a climber.  It just makes real climbers, those with training and proper safety gear, look very bad.

These morons are fairly high up, and walking on what appears  to be nothing but loose dirt and boulders.  I see helmets, rope, and other gear that can, and apparently should have, been used.  But it’s all clipped to their harnesses.  Pretty usefull there… NOT

http://www.break.com/index/two-climbers-avoid-near-death-accident.html