Last changed 7 July 2003
This document presents basic information about RAID for Linux, as presented to the Ogden Area Linux User Group on 6 February 2002.
This document will discuss what RAID is, what it is used for, under what circumstances one would want to use RAID. It will also give brief information about the demonstrations.
RAID is an acronym for Redundant Array of Inexpensive Disks. It is a set of related technologies normally used to protect data from hard drive failure (though it has other uses also).
All of the technologies distribute information across multiple hard drives while appearing as a single disk to the operating system. This "virtual" disk is often referred to simply as the "array." Because the disk array appears as a single hard drive to the operating system, external programs remain oblivious to the fact that data are scattered across multiple physical hard drives.
RAID resides beneath the file system layer, allowing the system administrator to use any file system desired. One may also partition the array in the same manner that one partitions a physical hard disk.
Every technology addresses a certain set of problems. This section describes the types of problems that RAID attempts to solve (the problem domain).
RAID addresses the problem of data loss due to hard drive crashes. Anybody who has experienced a hard drive crash knows the sinking feeling in the pit of the stomach that accompanies the realization that the hard drive is toast. If the data are valuable, and there are no external backups, the sinking feeling may rapidly switch to panic.
Data loss can be devastating. I personally know of two companies that folded due to complications that resulted from data lost from a hard drive crash.
RAID only addresses data loss due to hard drive failure. It does not address other sources of data loss, such as file system corruption that can occur with a power loss or a component failure.
Functioning disks that the RAID controller is using are said to be active. If one of the hard drives fail, the RAID controller will discontinue using the disk. We say that the disk has been deactivated.
If a hard drive fails, RAID will allow the computer to continue running without data loss in degraded mode. This means that the array will continue to deliver and store data, but there is no further redundancy. If another disk fails data will be lost. This is a very serious situation because data are typically scattered across multiple drives, making it difficult, if not impossible, for emergency data-recovery businesses to extract data from the hard disks. If the array is running in degraded mode, one ought to take immediate action to replace the defective drive.
When one replaces the defective drive, the RAID controller must copy data on it to bring the new disk up-to-date. This is called rebuilding the array. Once the new disk has been brought up to date, it leaves degraded mode and provides data redundancy. Thus, one's exposure to risk of hard disk failure endures from the time that a disk fails until the array has been rebuilt.
RAID's data protection compliments backups. It does not replace backups. This point emphasizes the fact that one should not relax one's data protection stance once RAID is in place. Though the preceding points make it clear that RAID addresses one area of vulnerability, human nature may induce one to become lax about protecting one's data from other dangers. Please do not do this!
If a RAID system is capable, failover compliments data protection, protecting the array from running in degraded mode.
One configures one or more drives to be spares. The spare disks do nothing but burn electricity until one of the active disks fails. When an active disk fails, the RAID controller shuts it down normally, but then automatically activates one of the spare disks to take its place. It immediately starts to rebuild the array. This narrows one's exposure to risk to the time period required to rebuild the array.
One still ought to replace the defective disk right away (which will become a new spare disk), but automatic failover reduces the urgency because the RAID controller automatically replaced it with a spare.
If a RAID system is capable, one may replace a failed drive without shutting down the machine. This feature, called hot swapping, becomes very attractive to high-availability servers, such as those found on 24/7 commercial web sites.
Normally one will use vendor-supplied software to disable the defective drive, allowing the drive to be replaced. Afterward one will use the software to activate the new drive.
Special hard drive containers allow one safely remove a hard drive without jeopardizing the hard drive controller. Without these containers, it is possible to destroy other hardware when connecting or disconnecting a hard drive. What was an annoyance now becomes serious trouble.
RAID allows multiple drives to be combined into a virtual disk larger than any single drive. This was very important in the past when large disk drives were prohibitively expensive. Even today with our multi-gigabyte hard drives, data warehousing needs have not gone away. If anything, the advent of terabytes of storage allows us the ability start new projects that need that magnitude of space.
Because RAID slaves multiple disks together, more data can be transferred than is possible with individual drives.
There are several methods that one may choose for distributing information across multiple hard drives. Each method approaches the problem from a different angle, and thus has its good and bad points. These methods are called modes or RAID levels.
Some contention exists among purists over the first two as they do not provide redundancy (the R in RAID), but they are grouped with the RAID family of technologies nevertheless.
This mode combines multiple disks into a single virtual disk. The virtual disk's capacity is the sum of the storage capacity of the individual disks.
As data are added to the virtual disk, the first disk is filled up, then the second, then the third, etc.
This "RAID" level provides no protection against data loss.
There is no performance gains for this mode on single reads and writes. If multiple processes access the array simultaneously, there will be a performance gain if the data resides on different physical disks.
Also known as striping, this is the first improvement over linear mode. Instead of filling one disk after another, it systematically breaks data into small chunks and distributes them over all disks equally. The virtual disk's capacity is the sum of the storage capacity of the individual disks.
This "RAID" level provides no protection against data loss.
Striping boosts performance for reads and writes. Performance increase is nearly linear for every disk in the array (as long as the bus' bandwidth does not become saturated). i.e. Four disks increase speed nearly fourfold under ideal conditions assuming the buses can move that much information.
Also known as mirroring, this is the first RAID level that provides data redundancy. Each disk in the array contain a complete copy of the data. This allows one to put multiple disks in the array for extra redundancy.
RAID-1 can withstand the failure of multiple disks (but not all disks, obviously).
Simultaneous reads can give improvement in speed because each read request can be directed to a different disk. Writes are slower as the data must be written to each disk in the array.
Because of the read benefits for simultaneous reads, one may add extra disks as active disks and not spares. Additionally, if disks are added as active disks and not spares, there is no need for rebuilding an array after automatic failover because all disks have complete copies of the data.
RAID-4 has been replaced by RAID-5. It dedicates one of the drives in the array to be hold parity information. Parity information allows data from a failed disk to be mathematically reconstructed.
Because parity information is stored on one disk, simultaneous writes are limited to the speed of writing to the parity disk.
RAID-4 can withstand the loss of a single disk, but not two. It does, however, provide for arrays larger than a single disk similar to linear mode and RAID-0.
RAID-5 eliminates the bottle neck of RAID-4 by distributing the parity information evenly across all disks.
As with RAID-4, it can withstand the loss of a single disk, but not two. It does, however, provide for arrays larger than a single disk similar to linear mode and RAID-0.
Known by several names, RAID-10 combines RAID-0 with RAID-1, requiring a minimum of four disks. It provides the data-protection benefits of RAID-1 while allowing for larger virtual disks.
| RAID Level | |||||||
|---|---|---|---|---|---|---|---|
| Linear | RAID-0 | RAID-1 | RAID-4 | RAID-5 | RAID-10 | ||
| Description | concatenation | striping | mirroring | dedicated parity disk | distributed parities | striping + mirroring | |
| Number of Disks | 2+ | 2+ | 2+ | 3+ | 3+ | 4+ | |
| Data Protection | No | No | Yes | Yes | Yes | Yes | |
| Array Capacity | sum | sum | single disk | sum of all disks but one | sum of all disks but one | sum÷2 | |
| Individual Disk sizes | any | same recommended¹ | same² | same² | same² | same² | |
| Performance | Single Reads | same | excellent | same | better | better | excellent |
| Single Writes | same | excellent | slightly worse | slightly worse | slightly worse | slightly worse | |
| Simultaneous Reads | same³ | excellent | excellent | excellent | excellent | excellent | |
| Simultaneous Writes | same³ | excellent | slightly worse | worse | slightly worse | slightly worse | |
The previous sections discussed the various methods of distributing information across disks. This section will discuss the two types of RAID controllers: hardware and software.
Commonly known as "hard" RAID, this method relies on dedicated hardware and firmware to control the arrays. Most RAID hardware comes as a PCI card, but some motherboards now include IDE controllers that can also act as a hard RAID controller.
Ordinarily the RAID card will provide a keystroke by which one can configure the arrays at boot time. Additionally, they normally work in tangent with user-level software for monitoring and configuring the hardware without shutting down the machine, thus allowing for high availability.
Hard RAID controllers typically provide extra circuitry to permit hot swapping.
RAID hardware controllers, like other disk controller, require a driver for the Linux kernel. Normally the driver exposes the arrays as generic SCSI devices. One partitions the arrays as one would any other hard disk.
Due to a multitude of technical reasons, "serious" RAID cards use SCSI disks. This paper does not endeavor to compare SCSI and IDE technologies, so suffice it to say that one ought not expect IDE to replace SCSI in the foreseeable future in critical servers.
Manufacturers have started to produce more IDE RAID cards, though that market niche has been shaky. For example, 3Ware recently discontinued their line of IDE RAID controllers. Fortunately customer demand persuaded them to re-establish their line of ATA-100 RAID controllers. Adaptec has flirted with IDE RAID controllers with varying commitment.
Some newer motherboards have RAID hardware built in. Linux support varies, so be sure to research these boards fully before buying one. Also be aware that these boards lack redundant hot-swappable features on more comprehensive (and expensive) motherboards for high-availability corporate servers. They make hard RAID more available to people, but cannot compete against high-end hardware.
In summary, you get what you pay for.
For the trivia-enthralled, in the quest for maximum performance, a handful of high-end RAID controllers track disk rotation and are able to synchronize order data reads and writes so that the commands are timed to coïncide with the disks' position. As the data passes under the disk head, the drive is ready to read or write data. Don't look for these at Walmart.
Also known as "soft" RAID, this method does not use any special hardware. The operating system provides a software layer that performs all of the RAID functions possible.
Note that the kernel cannot provide some functionality that strictly depends on hardware, for example:
Either of these reasons alone prevent the kernel from providing hot swapping. Thus to replace failed drives one must shut down the machine completely.
The Linux kernel exposes the arrays as block devices, similar to ordinary hard drives. Naturally there are some differences.
Given the drawbacks, why soft RAID? Besides the fact that it's a very interesting problem to solve, cost drives the soft RAID development. The kernel provides soft RAID for free. It allows common commodity hardware to have RAID capability without adding expensive dedicated RAID controller cards. It also provides a terrific learning environment for curious people on a shoestring budget.
| Hard RAID | Soft RAID | |
|---|---|---|
| Implementation | dedicated hardware | Linux kernel |
| Automatic Failover | yes | yes |
| Hot Swap | yes¹ | no² |
| Cost | hundreds of dollars | none |
| CPU Impact | negligible¹ | typically 5-15% |
This section of the document contains notes for the demonstrations.
My workstation contains an Escalade RAID card. This particular model provides support for two IDE drives operating in RAID-1 mode.
IFM Services, LLC's primary file server contains a versatile four-disk controller. One may configure this controller for any RAID mode. Escalade also supplies an eight-disk controller.
The demonstration will only involve the two-drive IDE controller to give participants a feel for a hardware RAID controller.
Software RAID revolves around a configuration file and commands to create, start, and stop a soft RAID array.
