What is RAID?

[Zephik]

I've tried Webopedia, Wikipedia and a few other websites that have attempted to explain what RAID is, but I just don't get it. What is RAID?? I hear all of this talk about Striping and Mirroring and it just goes through one ear and out the other. Could someone explain what RAID is and what these terms are in monkey talk? Just get me to understand what it is and what its used for, then if you want you can go ahead and get technical with it.

Just for your information, I'm asking because the motherboard I am getting has RAID, which is something I've never had before. It has 0/1/0+1/5... whatever the heck that means. lol

[Bucko]

RAID is using more than one HDD at once.
It could be using two HDD and it writes and reads top both at once, sharing the data, effectively doubling your read/write speed.
However, if one drive dies, you lose ALL your data.

Or RAID can write the data to two dives at once, so if one drive dies, you still have an exact copy of that data on the other drive.

From there, there is a combination of them. That is the basics though.

[Bucko]

Here's a post I was looking for:


RAID 1 (redundancy) and RAID 0 (speed) use 2 drives

RAID 10 and RAID 1+0 uses 4 drives

Check out the different configurations. (http://www.9to5computer.com/atto/Comparing-Raid-Configurations.htm)

This post is perfect with my screen name.

[Zephik]

Well, I only have one hard drive. So is RAID pretty much useless unless I have two or more hard drives?

[Bucko]

That is correct. You need more than one drive.

[OvRiDe]

The link Bucko gave is pretty good about telling the different types of RAID configurations. I have had alot of experience with RAID controllers and taken several courses from storage manufacturers. So I will try to shed a little light that helped me on the RAID concept itself.

First RAID stands for Redundant Array of Independent Disks.

RAID 1 is the most basic of raid. Everything written to one drive is also written to another drive, thus also called mirroring. Not much more to say about that.

The rest really boils down to what we call Striping. What that means is a portion of the data is being written to multiple disks. Since everything stored on a hard drive is essentially bits (1's and 0's) it takes the bits and sequentially writes them to the multiple drives one after another, thus "striping" the data across X number of drives. This is known as RAID 0, there is no redundancy or recovery for this type of data. So if you loose 1 drive, the data is then incomplete and therefore unreadable.

So know your saying .. thats not cool.. Sure it may be fast but it seems kind of risky! Your right. Thats when the next levels of RAID come in. Again the most basic is to mirror your RAID 0 stripe set. This is known as RAID 0+1 or some like to call it RAID 10. All that means is you Mirror your stripe set to an equal number of drives. ie. If you have a 3 drive RAID 0 stripe, it would mirror it to 3 other drives making a total of 6 drives.

So.. you say I don't want to use that many drives and only be able to use half the capacity. Well thats where the "Stripe sets with parity" comes into play. Parity brings a way of recovering the data if one of the stripes is lost. How does it do that?? Its freaking MAGIC.. no not really.. Its actually quite simple. A parity bit allows you to recreate the data that is missing.

For example lets say you have 3 drives. If Drive 1's corresponding bit is 0 and Drive 2's corresponding bit is 0 then Drive 3 (the parity drive, more on that later) would be 1.

This is all basic binary arithmetic. Which dictates the following

0+0=1
0+1=0
1+0=0
1+1=1

So with the information we have on all 3 drives we can alway rebuild a missing set of the stripe.

Lets say we loose Drive 2. Since we know that Drive 1's bit was 0 and that the parity bit on Drive 3 is 1 we can deduce that:

0+X=1 and we know that 0+0=1 so Drive 2's bit would have to be 0. So then when the drive is replaced, the RAID controller can rewrite the drive using the information from the other 2 drives. At this point it doesn't matter which drive is lost, you always have the information needed to rebuild it. Its not as fast as RAID 0 because you still have to read and write the parity bit, but its a lot safer, as long as you don't loose more then 1 drive at a time. Then you are back to not having enough information to rebuild the data.

Once you have gotten to the "Stipe set with Parity" there are 2 basic flavors. You have RAID 3 and RAID 5. I won't go into to much detail there, because the link that Bucko and Redundant posted covers that. It basically boils down to RAID 3 uses a dedicated parity drive and RAID 5 spreads the parity bits evenly across the stripe set.

I know this has been a bit of a long read but I hope it helps shed some light on the basics of RAID sets.

[Zephik]

Well thats lame.

Way to drop the disappointment bomb on me there Bucko! geeze, haven't you ever heard of lying?

LoL

[The Black Pumpkin]

It helped me, OvRiDe.

That was a very good "short" guide! ;)

+rep

[Bucko]

geeze, haven't you ever heard of lying?


Sorry.

<lie>you can use one drive, but you have to overclock the HDD by raising the voltage to at least 450v. You also need to underclock the cooling fans and raise the FSB of every second USB port.</lie>

[Zephik]

Thanks OvRiDe! That cleared so much up in my head, it all makes alot more sense now!

...dangit, now I want to go buy some more hard drives... lol

[Spawn-Inc]

so let me see if i have this right... lets say i go buy 4 hdds 250gbs each. if i took drive 1 and 2 in a raid 0 setup could i raid 1 it with drive 3 and 4 which would also be in raid 0. would that work? and i would end up with 500gbs of space or 1tb?

[OvRiDe]

If you had 4X250GB drives. When it comes to mirroring its a 1 to 1 ratio.. So you could essentially have 2X250GB Volumes, OR if your RAID controller supports RAID 0+1 you could stripe 2 of the 250GB into a 500GB Volume and then mirror that to the other 2 Drives. So you would have 1 500GB Volume using 4 250GB drives. The other option would to use a RAID 5 type setup in which you would stripe all the drives together which would equal 1TB but you must subtract the parity bits, so I believe that would equal about 1/3 of the total capacity. Which would leave you with aprox 750GB's of redundant storage space.

[The Black Pumpkin]

Can you take 5 250's, 4 for the 1TB of raid 5, and the other fifth one for the parity bits?

And if I'm not mistaken, if you stripe two drives of equal value, windows shows one drive, with total capacity including both drives? In other words, 2 100 gb would be a 200 in windows, at double speed? Or did I read your last post wrong...

[danthegeek]

Would a gamer benefit from raid 0? Faster load times, less load lag in Oblivion, etc?

EDIT- also is there a way to setup a raid 0 without loosing the existing data on your Hd?

[OvRiDe]

Can you take 5 250's, 4 for the 1TB of raid 5, and the other fifth one for the parity bits?

And if I'm not mistaken, if you stripe two drives of equal value, windows shows one drive, with total capacity including both drives? In other words, 2 100 gb would be a 200 in windows, at double speed? Or did I read your last post wrong...

Lets just say sorta.. That would be a RAID 0. In a RAID 0 configuration, since there is no parity you get the full sum of the drives capacities. When it comes to speed there are a lot of factors involved. It all depends on how you have your drives configured and the interface that is being used.

This can get really complicated so I will try to keep it as simple as possible. Since I know more about SCSI controllers I will use it as an example.

SCSI Adapters have channels and IDs. A channel is composed of multiple IDs. The Adapter can read/write to multiple channels simultaneously, but it can only read/write to the IDs on that channel one at a time. So if you stripe your data across multiple channels it becomes very fast because all the data can be written or read all at once. But if you drives are all configured on the same channel across multiple IDs, the controller still has to write/read to them one at a time, and therefore some of the speed is lost.

I haven't done a lot of research into how SATA controllers handle data, IE. whether they can read/write to multiple drives simultaneously, but I figure they can.

When it comes to using Windows to stripe the drives, that is called Software RAID and it is RAID 0, or it can also Mirror as well. So you are correct in windows 2 100GB drives would become 1 200GB Volume. In my previous posts I was referring to hardware RAID configurations, which require the controller to perform the striping or mirroring, but the concept is the same for software RAIDs as well. Make sense?


Would a gamer benefit from raid 0? Faster load times, less load lag in Oblivion, etc?

EDIT- also is there a way to setup a raid 0 without loosing the existing data on your Hd?

Yes, with the proper configuration and fast drives, you will see a significant reduction in load times on some games.

as for your second question.. Unfortunately no. In most cases whether its software or hardware RAID, when the RAID array is initialized all data is lost, and you have a bright shiny new Volume ready to be partitioned and formatted. :(

[Spawn-Inc]

If you had 4X250GB drives. When it comes to mirroring its a 1 to 1 ratio.. So you could essentially have 2X250GB Volumes, OR if your RAID controller supports RAID 0+1 you could stripe 2 of the 250GB into a 500GB Volume and then mirror that to the other 2 Drives. So you would have 1 500GB Volume using 4 250GB drives. The other option would to use a RAID 5 type setup in which you would stripe all the drives together which would equal 1TB but you must subtract the parity bits, so I believe that would equal about 1/3 of the total capacity. Which would leave you with aprox 750GB's of redundant storage space.

so it sounds like it would do what i want. if lets say drive 1 fails then you could replace it with drive 3 right? then you would be left with a regular raid 0 setup + the 1 extra drive?

also +rep for heling with raid stuffs!

[OvRiDe]

Im sorry but I am not sure which scenario you are referring to. But once a drive becomes part of a RAID set there is no going back with the exception of RAID 1 or mirroring. In some cases you can "break" the mirror and you will have 2 drives that are exactly the same. When it comes to a stripe set, if you remove a disk from the set, you are vulnerable to data loss (RAID 3 and 5) or in the case of RAID 0 your sunk. On high end systems you can have "HOT" spares which are basically blank drives that hang out until a drive fails. Once it detects a failure the controller will automatically rebuild the missing data to the hot spare, which gives you an extra layer of protection. The best and only real solution when it comes to loosing a drive is to replace the bad drive as soon as possible.

Hope this answers your question, if not let me know and I will do my best.

[danthegeek]

Thanks OvRiDe +rep. I may have to give this a try. (sorry of I hi-jacked)

EDIT- Said I have to "spread it around" sorry But A personel +REP I can give :)

[OvRiDe]

Thanks OvRiDe +rep. I may have to give this a try. (sorry of I hi-jacked)

EDIT- Said I have to "spread it around" sorry But A personel +REP I can give :)

Hijacked? Absolutely not! To me thats what this is all about, asking questions and using our collective knowledge to help improve ourselves. I am just glad I was able to contribute! And as for the rep Thankyou very much .. although its nice to see the recognition ;) , its not why I am here. I am just glad to help out where I can.

EDIT: The personal rep means more to me then any green box :D!

[nil8]

OvRiDe, you're all over this. Good job. rep+

Want to explain the difference between hard and soft raid?

[OvRiDe]

OvRiDe, you're all over this. Good job. rep+

Want to explain the difference between hard and soft raid?

Absolutely.. Its pretty basic.

Hardware RAID is controlled by the Drive controller itself, thus the name hard or hardware RAID. Whether its SCSI, IDE, or SATA, there is software embedded in the controller to handle the the functions of the RAID. The nice thing about this is when it gets to the OS, the OS will just see it as it would any other disk. Hardware RAID controllers usually include extra RAM to help increase performance and provide extra stability and reliability.

Software RAID is where Software most likely the OS is controlling the RAIDed drives. This is very handy when you don't have a controller with the embedded Hardware to do RAID. Unfortunately I believe you are pretty much limited to RAID 0 or RAID 1 when dealing with Software RAID configurations.

[Drum Thumper]

I would like to point out that to get full benefits from running in RAID (ie 0,1,3,5), you're going to need Win2k Server or Win2k3 Server editions. The XP Pro and Win2k both only support RAID 0.

Of course, you can bypass that little glitch with 3rd party software.

HTH

[OvRiDe]

I would like to point out that to get full benefits from running in RAID (ie 0,1,3,5), you're going to need Win2k Server or Win2k3 Server editions. The XP Pro and Win2k both only support RAID 0.

Of course, you can bypass that little glitch with 3rd party software.

HTH

This is only the case for Software RAID, when it comes to Hardware RAID the OS is oblivious to the fact that is not a single disk volume. No offense I just wanted to make sure that was clarified and didn't confuse anyone.

I apologize, but it appears that I was inaccurate in my statement that Software RAIDS only support RAID 1 and 0. It does appear that both Linux and Windows (including XP) will do Software RAID 5. Here (http://www.tomshardware.com/2004/11/19/using_windowsxp_to_make_raid_5_happen/) is an article from THG where they explore RAID 5 in Windows XP. I found the last page with the performance charts quite interesting. It demonstrates the performance difference between Hardware RAID setups and Software RAID. I rarely use software RAID, so I probably should have checked before I spoke. :redface:

[DaveW]

Redundant Array of Independent Disks.

Actually, it's Redundant Array of Inexpensive Disks. If you think about it logically, independent disks doesn't even make sense. :p

-Dave

[Airbozo]

Then why do you set up your raid controllers (F6) when XP is installing? That would, to me at least, signify that you can get the full beneift of any RAID config out of more than Server 2k or 2k3..

-J

You are correct Minty! For example, My motherboard at home supports raid on the IDE bus. I have setup my system with a striped raid setup for speed (not that significant if you ask me). The _mother_ board (or more correctly, the IDE controller) handles all functions of the raid subsystem _regardless_ of what OS I am booted into.
\

EDIT: BTW this site has a really good graphical animated explanation of different raid levels and how data is transfered;
http://www.acnc.com/04_00.html

[OvRiDe]

Actually, it's Redundant Array of Inexpensive Disks. If you think about it logically, independent disks doesn't even make sense. :p

-Dave

Actually. both are right. Its one of those places where some use independent and some use inexpensive.

http://searchstorage.techtarget.com/sDefinition/0,,sid5_gci214332,00.html
http://www.answers.com/topic/raid-technology
http://www.pcmag.com/encyclopedia_term/0,,t=raid&i=50148,00.asp

The last link seems to sum it up the best..


In the late 1980s, the term stood for "redundant array of inexpensive disks," being compared to large, expensive disks at the time. As hard disks became cheaper, the RAID Advisory Board changed "inexpensive" to "independent."


Then why do you set up your raid controllers (F6) when XP is installing? That would, to me at least, signify that you can get the full beneift of any RAID config out of more than Server 2k or 2k3..

-J

I just want to make sure to point out that when you it asks you to press F6 for RAID controllers, it is so you can load a driver for a hardware raid controller. Drumthumper was referring to software RAID. Again when you load a driver for a Hardware RAID controller, the OS does not know that there is more then one drive attached to the Volume, because all of that is being handled by the hardware. Make sense?

[Drum Thumper]

Actually, it's Redundant Array of Inexpensive Disks. If you think about it logically, independent disks doesn't even make sense. :p

-Dave

I've seen it both ways. In fact, on page 326 of the All-In-One CompTIA A+ certification guide, it's list as both in the same sentence. So it's really just a matter of preference.

The Independent Disk monkier came from the days when disks were pricey. The RAID concept is over 20 years old, and storage prices have plummeted since then, so looking at it that way, Redundant Array of Independent Disks does make sense.

[Drum Thumper]

This is only the case for Software RAID, when it comes to Hardware RAID the OS is oblivious to the fact that is not a single disk volume. No offense I just wanted to make sure that was clarified and didn't confuse anyone.

I apologize, but it appears that I was inaccurate in my statement that Software RAIDS only support RAID 1 and 0. It does appear that both Linux and Windows (including XP) will do Software RAID 5. Here (http://www.tomshardware.com/2004/11/19/using_windowsxp_to_make_raid_5_happen/) is an article from THG where they explore RAID 5 in Windows XP. I found the last page with the performance charts quite interesting. It demonstrates the performance difference between Hardware RAID setups and Software RAID. I rarely use software RAID, so I probably should have checked before I spoke. :redface:

My bad. Shouldn't really be posting @ 3 am when I know I should be in bed. Yes, I was referring to Software RAID. Looks like you pretty well have the class straightened out!

[Redundant]

Actually, it's Redundant Array of Inexpensive Disks. If you think about it logically, independent disks doesn't even make sense. :p

-Dave

It should be changed to Redundant Array of Interconnected Drives. That makes the most sense to me. :up:

[XcOM]

i prefur JBOD!!
My old server used mirrored, i used it for stabilty and backup, mainly coz i transfur LOADS of data, Alot of the time, so i want backups, i've only had to restore the drive twice.

[Drum Thumper]

JBOD! I was going to post about JBOD, but figured that this was the RAID thread, so...

Just a Bunch Of Disks. Functional, but what happens if, say, your drive that's holding all your modding pictures goes belly up?

I'll take RAID any day of the week, tyvm.

[progbuddy]

Harddrives are fairly cheap now. I bought a Western Digital Caviar SATA II harddrive for 43 bucks of newegg. Shouldn't put a big hole in your wallet to go buy one.

[Airbozo]

i prefur JBOD!!
My old server used mirrored, i used it for stabilty and backup, mainly coz i transfur LOADS of data, Alot of the time, so i want backups, i've only had to restore the drive twice.


JBOD! I was going to post about JBOD, but figured that this was the RAID thread, so...

Just a Bunch Of Disks. Functional, but what happens if, say, your drive that's holding all your modding pictures goes belly up?

I'll take RAID any day of the week, tyvm.

I have a JBOD sitting right next to me, connected to my Octane2 that has 10 drives, software raid 0.

Oh and I pre-date most of you and when I was taking my UNIX tests several decades ago, the ONLY correct answer was; Redundant Array of Inexpensive Disks. Some electronic terminology has become out dated. Just like SCSI; Small Computer Systems Interface. Some of the systems that rely ONLY on scsi are far from small...

[nil8]

RAID is called inexpensive because it used cheaper formats for work.
Much, much cheaper than old big iron stuff from the 70's and 80's. I'm sure bozo could fill you in on this much better than myself.

In modern times, it's called RAID and it doesn't matter which version you prefer. What matters is that it works, and works well.

Is there a difference between jbod and a regular raid setup or is it just another way of saying raid?

I'm surprised that no one mentioned that mobo-integrated raid is almost always soft raid. It relies on the processor to do it's number crunching. If you're talking big raid 5 or 50 arrays, that's some serious processing.

Hard raid is almost always a separate card running PCI 64 bit or PCI-X.
The most common units that I know of use XOR processing. Could someone elaborate on this?

Modern raid cards come in different interface and configuration flavors. From a cheap 20 dollar 0/1/10 pata 100 PCI card to a 1000 dollar 0/1/3/5/10/50 12 port sata 2 setups.

[Airbozo]

RAID is called inexpensive because it used cheaper formats for work.
Much, much cheaper than old big iron stuff from the 70's and 80's. I'm sure bozo could fill you in on this much better than myself.

One of my first real computer jobs was fixing disk drives. 500mb drives that took 2 people to lift and measured 19" wide, 12" high, and 24-30" deep. A refurbished one cost $20k. I know, I knocked one off of a cart and could not fix it. I heard about that for the rest of the time I was there (Amoco Production in Denver). We used to replace the heads, motors logic boards and platters. It took about 3 hours and an ocilliscope to re-align it after swapping any parts. The bottom layer of the bottom plater was nothing but an alignment track. I also fixed floppy drives. Head replacement, motor replacement, calibration. It was cheaper at the time to repair the drive instead of buying a new one.

In modern times, it's called RAID and it doesn't matter which version you prefer. What matters is that it works, and works well.

Is there a difference between jbod and a regular raid setup or is it just another way of saying raid?

Typically when referring to a JBOD vs RAID enclosure, the RAID enclosure had the necessary onboard logic for RAIDing the drives, whereas the JBOD did not (although you could hook it up to an external RAID device).

I'm surprised that no one mentioned that mobo-integrated raid is almost always soft raid. It relies on the processor to do it's number crunching. If you're talking big raid 5 or 50 arrays, that's some serious processing.

Sort of but not quite. Take for instance the raid setup I mentioned at my house. It is completely transparent to the OS that there is a RAID device. The motherboard handles everything. There is however a driver that has to be loaded into windows to recognize the device (F6). None of the raid logic is done in windows, so it really is hardware raid. HOWEVER, most of the add in raid cards (the expensive ones anyway) have their own processor and memory on board to handle ALL the raid functions, taking the load off of the onboard CPU. Now I am curious if there is a performance hit on the processor to handle the raid stuff. I know there logically has to be, but how much? I have been itching to do some raid testing with both sata and scsi drives to determine thew speed boost, once I am done with the next cluster installment and have the Modders Challenge underway I will have to test this. I have access to a machine that can handle the 64 bit raid cards and have a low end and a high end card here at work.

Hard raid is almost always a separate card running PCI 64 bit or PCI-X.
The most common units that I know of use XOR processing. Could someone elaborate on this?

You got me on that one....

Modern raid cards come in different interface and configuration flavors. From a cheap 20 dollar 0/1/10 pata 100 PCI card to a 1000 dollar 0/1/3/5/10/50 12 port sata 2 setups.

...you forgot the $2k SAS raid card... ;)

[OvRiDe]

I'm surprised that no one mentioned that mobo-integrated raid is almost always soft raid. It relies on the processor to do it's number crunching. If you're talking big raid 5 or 50 arrays, that's some serious processing.

Like AB said, If there is hardware present to handle the RAID logic, then it would be hardware RAID. I mean if you really come down to it ALL RAID is software. Since there isn't a mechanical device separating the bits to each drive, its software that does the work. The difference is where the software is being executed from. If there is a dedicated device that controls the RAID logic its hardware, if it is controlled by the OS or some program that is loaded into the OS then its Software RAID.


Hard raid is almost always a separate card running PCI 64 bit or PCI-X.

Those are most common today, but RAID controllers were also available in 16 bit ISA/EISA slots and of course 16 and 32 bit PCI slots as well. RAID technology was first developed in 1987 as software and later in '93 the development of Hardware RAID devices began.


The most common units that I know of use XOR processing. Could someone elaborate on this?

XOR processing is basically what I explained in the first post about parity. I was trying to keep it really simple. Paritly actually uses a subset of binary artithmatic called Bitwise Operation. So where I said:


0+0=1
0+1=0
1+0=0
1+1=1


Really its

0 XOR 0 = 1
0 XOR 1 = 0
1 XOR 0 = 0
1 XOR 1 = 1


HOWEVER, most of the add in raid cards (the expensive ones anyway) have their own processor and memory on board to handle ALL the raid functions, taking the load off of the onboard CPU. Now I am curious if there is a performance hit on the processor to handle the raid stuff. I know there logically has to be, but how much? I have been itching to do some raid testing with both sata and scsi drives to determine thew speed boost, once I am done with the next cluster installment and have the Modders Challenge underway I will have to test this.

In the post below,the link to THG has some perfomance charts toward the end of the article, that shows the performance of several RAID controllers as compared to the Software RAID used by Windows XP. I would be curious to see actual charts of the CPU usage during write and read operations, as well as during the rebuild process.


Here (http://www.tomshardware.com/2004/11/19/using_windowsxp_to_make_raid_5_happen/) is an article from THG where they explore RAID 5 in Windows XP. I found the last page with the performance charts quite interesting. It demonstrates the performance difference between Hardware RAID setups and Software RAID.

If you get a chance to do that testing, I would love to see the results.

[DaveW]

Great stuff here man.

-Dave

[XcOM]

i like JBOD on my system maily becuase its my main machine, I have no use for fancy setups, No need for backups, and its fast enough for what i do, theres no Vital data stored on it, ON CD-Images (BACKUPS OF COURSE) of which the orgionals are in a wallet, to prevent damage, Random Downloads and my OS, all on 3 HDD's,

Its strange, theres setup in JBOD eventhough there on a "Promise Ultra100 TX2" raid card, Wonderful cards, and i did see a peformace JUMP from onboard IDE to the raid card. My server holds all my documents, music and other stuff, everything thats important and i can't be arsed to store on my machine.

And to answer your question, what is one failos in JBOD, simple, use a disk recovery (Provided its still detectoed by bios) and recover, and replace the drive.


Never had to do that on my machine!