Gerald Brandt

Stay-at-home Dad/Writer/Computer Programmer  

“Writing a novel is like driving a car at night. You can only see as far as your headlights, but you can make the entire journey that way.” — E. L. Doctorow

Archive for category Computers

Drobo Speed vs Linux RAID5 Speed

Feb 25

Posted by gbr in Computers | 4 Comments

I recently had a failure on my 4 TB RAID5 array.  It happened when I moved the server from the rack to my office, in preparation for converting it to a desktop system.  It would still host the RAID array, but it would no longer be running all the virtual machines.

During the move, I also upgraded the RAM.  Shortly after powering the server on, I noticed that any access to my harddrives (RAID or not) was extremely slow.  The kernel showed high load, with 98% being in system.  Wow.  Before I had time to realize it was the RAM I added, I lost a drive on the RAID array.  I pulled the RAM, and things went back to normal.  I added a new drive and started rebuilding the array.  At 97.1% complete a second drive got a permanent read error.  Good bye array.

I managed to get the array back up degraded, but I couldn’t rebuild it.  Now I know I could probably have used dd to copy the disk to a new one, get another new drive, and rebuild that way.  Oh well, hindsight is 20/20.

I needed to get my data off the degraded array.  There was no critical data on it, so it had never been backed up.  How am I going to back up 2+ TB of data anyway.  It mostly contained over 1000 recordings from MythTV, some audio and video, and pictures.  Still, if I could save the data, I really wanted to.  So, I went out and purchased a Drobo.  Drobo’s don’t really like Linux that much, so I hooked it up to my Mac Mini and started copying data over via rsync.  Wow, was that slow!

It turns out the Drobo is a really. slow. device.  Although it can handle watching live TV via MythTV, it barely keeps up.  Anything else done on the Drobo kills the TV.  Granted, the Drobo is a Firewire 800 device, but it still slower than that would imply.  Knowing that I will be recording and watching 2 HD streams at the same time, I knew the Drobo wouldn’t cut it for me.  For example, a Drobo rebuild could take days, and it’s all internal to the Drobo, no firewire used.  But, during rebuild, live TV was impossible to watch.

I built a new 4 disk 2.8 TB RAID5 array under Linux, and started transferring live TV and recording to the RAID5 array.  Things got much better  I decided to do some tests.

The Drobo, connected via Firewire 800, writing a 16GB file took 628.89 seconds (10.48 minutes) ~25 MB/s
The Linux RAID 5, writing a 16GB file took 135.59 seconds (2.26 minutes) ~121 MB/s

25 MB/s is doable for two HD channels, except for the issue that nothing else could use the drives at the same time.

I then tested the connection via the network.  My MythTV Server is a simple Pentium M 1.5 GHz system, and it uses remote Samba shares for it’s storage, over GB ethernet.

The Drobo via OSX SMB share, same 16 GB file took 1451.37 seconds (24.19 minutes) ~ 11.3 MB/s
The Linux RAID5 via Linux SMB share, same 16 GB file took 459.08 seconds (7.65 minutes) ~ 35.7 MB/s

Ummmm.  Ouch!  The Mac Mini is a Core2Duo at 2.6 GHz, the Linux is a Core2Quad at 2.6 GHz.  Neither used any noticeable CPU during the file transfer.

Tweaking the OSX smb.conf file didn’t help any.  What did help was specified here.  That alone changed my speed to 16 MB/s.  Still not good enough, but better.

The Drobo is a cool device, and has it’s place.  But not in  a media streaming type environment.  I’ll use the Drobo to store my audio, pictures, and movies, but not my live TV recordings for MythTV.

Tags: ,

Jan 15

Posted by gbr in Computers, Writing | No Comments

A good friend of mine, Rob Peters, posted in his blog some pearls of wisdom he gleaned from Jennifer Kornelsen, Ph.D. :

1)  The simplest explanation is the best. (i.e. the most likely, the most accurate, the most truthful)

2)  The data is what it is. (trust it, let it be…)

3)  If you’re nervous and think you’re going to puke, eat something colourful! (at least then it will be Spectacular!)

Rob found these pearls to be as true in Design as in Neurophysiology.  I’m here to say they also apply extremely well to Software Design/Development, and IT work.  It’s unfortunate that, in software and IT at least, a lot of people forget item number one.

However, in the writing of fiction, I’m not so sure (well, except for item 3), at least not on the surface.  For item 1, when the reader reaches the end of a book, the final explanation of event should be clearly evident and obvious, and yes, even simple.  But during the reading of the book, the simplest explanation of the events occurring is usually the one you want the reader to follow, but should not be the true reason.  You gotta keep ‘em interested.

In writing fiction, item 2 closely correlates to item 1.  This can be especially true when reading a first person narrative, where everything presented to the reader is viewed through the eyes of a single (or potentially multiple first person) characters.  If the character looks at the world through rose colored glasses (cliché), then that is how the reader will interpret the events (data) in the book, and therefor, the data is tainted.

Perhaps I should re-phrase my original statement.  To the reader of fiction, the above pearls should not be true, but to the writer, they probably should.

A day away…

Sep 13

Posted by gbr in Computers, Writing | No Comments

I had a wonderful opportunity today.  I was able to take off and spend the day alone.  I brought my laptop, worked on The Courier and started working on the undo/redo code for my WordStar clone: WordTsar (though I may change that name).

I also brought some free range chickens to Ev and Rob in Winnipeg Beach, and spent some time on the shores of Lake Winnipeg.  A wonderful and restful day.  Thanks Marnie!

beach

iPhone Fail…

Aug 5

Posted by gbr in Computers, misc | 2 Comments

iphone-failI’ve had my iPhone for quite awhile now, and I’ve been adding and removing apps as I needed them.  Before each removal, I synced the phone, and made a backup of the device, just to be sure.

Well apparently that’s not enough.  First, syncing the phone does not, by default, sync the apps that are on the phone.  Okay, that’s my bad.  I should have looked at all the other tabs in iTunes, and figured out what was going on.  Now that I have ’sync apps’ checked, they still don’t sync since my Mac doesn’t have rights to the apps.  Great, thanks.

However, backup doesn’t actually backup the phone.

My phone had been getting slower and slower as time went on.  I did some research, and apparently a reset of the phone would fix the issue.  So I dutifully did a backup of the phone, did a full reset, and then a restore of the phone.  BLAM!  I lost almost everything.  The only thing that stayed was my email and calendar settings (oh, and my contacts list).  Everything else was gone.  Wiped out.  Not to be seen anywhere.  That included, of course, my handy dandy ‘Password Wallet’ that stored all of my passwords for the various systems I use and admin.

I’m not sure I have a copy of those passwords.

The iTunes App store keeps a list of the software I’ve downloaded, so I’ve restored all my purchased apps.  But c’mon.  Really?  A backup that doesn’t actually backup?  What the hell are you guys thinking?

Now some Apple dude (or dudette) is probably going to tell me what I did wrong, and why backup does what it does (my guess is something to do with DRM bullshit).  But that’s not going to bring back my lost data, or the lost time I spent getting my apps back (and I’ve only gotten back the paid apps so far).

All-in-all, I’m not a happy camper.

edit: But my iPhone is fast again!  LOL

Tags:

iPhone OS 3.0

Jun 17

Posted by gbr in Computers, misc | 2 Comments

The new software for the iPhone is available today.  iPhone 3.0.  Get your copy today!

iphone_home

edit: 8:40 AM – I just tried to update, and it’s not available yet.

edit: 4:18 PM – Apples servers are being hit hard, and I can’t authenticate for the update.

edit: 4:47 PM – Update in progress.

Tags:

2008 Accomplishments

Jan 6

Posted by gbr in Computers, Parenting, Writing, blog, misc | 2 Comments

I figured I’d write a post about what I accomplished in 2008.  This is it.  I’ll also add what I hope to accomplish with each item in 2009.

WordSlinger Beta released.  WordSlinger is an Integrated Development Environment for writers.  It’s geared towards the way I write, although I’m looking at integrating some users wishes into it.  In 2009, I’m hoping to bring this up to release quality and get it out into the world.

Moved my computer systems from Gentoo based servers to Ubuntu, as well as my desktops.

Got interviewed as a stay-at-home dad for Sharp magazine

Went to Ghana, Africa for two weeks.  (Yes, it was an accomplishment)

Went to the World Fantasy Convention for the first time.  I’d like to go again in 2009.

Started updating the blog weekly with a list of links I’ve found interesting.

Completed The Courier and got a couple of copies out to first readers.  I’ll touch it up, work on a Query, and get it out to agents in 2009.

Stopped being a true stay-at-home dad.  I accepted a job that still allows me to be the primary caregiver to my kids (essentially I’m home whenever the kids are).  I was lucky to find a forward thinking company.  My work load has increased a lot, but so far so good.

BuildMaster Beta released.  BuildMaster is an application that does nightly builds of software projects.  It notifies users on whether the builds succeeded or failed, and if they failed, what the issue was.  For 2009, I’d like to get this finished with a nice web page to report status, and some sort of data storage on the backend to show the history of builds.

Boy, when it’s listed out like that, I feel like a pretty lazy guy.  Where did all the time go anyway?

What are some of your accomplishments this year?

As the blog languishes

Jul 9

Posted by gbr in Computers, Parenting | 1 Comment

To my faithful reader (singlular),

I apologize for the lack of posts.  I’ve had a contract job since early this year, and on top of putting in 6+ hours a day on that, I’ve also kept up my stay-at-home dad responsibilities.  This week, I’m doing 16 hrs a day on the contract, and Grandma is taking care of the boys.  Thanks Grammy!

Next week I’m of to Albequerque to visit my clients clients, and hopefully make a delivery.  The week after that is 3 weeks of camping with my boys, and a chance to rest.

I’ll try to post something sooner next time.

Sincerely,

Gerald

Where has the month gone?

Jan 18

Posted by gbr in Computers, Writing, blog | 3 Comments

Here it is, well into the month already, and I’m not sure where the time went.

Let’s see what I did:

  1. A lot of work on WordSlinger. I broke it adding folders to the draft/scene area, but it’s almost working now.
  2. Tore apart our bathroom and started rebuilding it. The previous owners of the house glued linoleum 4 feet up the walls, and trying to take it down destroyed the plaster and lathe walls. Doh. I also found rotten wood and some mold. Not nice.

Well, that’s it. Two items. Granted, both took a fair amount of work, and neither are finished, but still… Time to get back to a regular schedule.

Web Pages Move

Jan 6

Posted by gbr in Computers, blog | 3 Comments

I’ve moved this web site from a Gentoo based/Apache 2 install to a new Ubuntu Server/LiteSpeed install. We are experiencing some difficulties with the transfer. If you see anything wierd not already mentioned in the comments, please leave a new comment with the particulars.

I’m still running on a virtual system, but response times should increase dramatically.

WordSlinger 0.1.0

Jan 2

Posted by gbr in Computers, Writing | 4 Comments

Well, after a few mishaps and backtracks (yes, 2 years of being a stay at home Dad can effect ones software development skills), I've decided that my latest application is ready for Beta testing.  It took a bit longer than I thought to get to this point, but that's the way it is.

I'm now moving my current WIP into it, and we'll see how it goes.  I really wanted to get snapshots into it, but that can wait a bit. 

Watch the Video: wordslinger-010.htm

 

 Images:

 

 
 

 

6 visitors online now
6 guests, 0 members
Max visitors today: 7 at 02:40 am UTC
This month: 20 at 03-03-2010 02:30 am UTC
This year: 48 at 01-13-2010 01:32 pm UTC
All time: 48 at 01-13-2010 01:32 pm UTC