Linux: Why software RAID?
Menu:
The "software or hardware RAID?" question comes up often. I think I answer the question at least once every six months or so. And the responses to such a question are often filled with, in my opinion, inordinate praise of expensive hardware RAID solutions, and inordinate scorn for software RAID. This page is my attempt to explain a Linux kernel engineer's point of view on the matter.
NOTE: This comparison excludes SAN and other external RAID solutions. Externally attached storage is outside the scope of this discussion. Externally connected solutions can obviously be SAN, software RAID, hardware RAID, or a combination thereof.
Why prefer Linux software RAID?
- Potential for increased hardware and software biodiversity
- Kernel engineers have much greater ability to diagnose and fix problems, as opposed to a closed source firmware. This has often been a problem in the past, with hardware RAID.
- Disk format is public
- ...thus, no vendor lock-in: Your data is not stored in a vendor-proprietary format.
- A controller-independent, vendor-neutral layout means disks can be easily moved between controllers. Sometimes a complete backup+restore is required even when moving between hardware RAID models from the same vendor.
- Eliminates single-points-of-failure (SPOF) compared to similar configurations of hardware RAID.
- RAID5 XOR runs on host CPU, which practically guarantees that it is far faster than most hardware RAID microcontrollers.
- RAID5 XOR speed increases as host CPU speeds increase.
- RAID speed increases as host CPU count (multi-thread, multi-core) increases, following current market trends.
- Cost. A CPU and memory upgrade is often cheaper and more effective than buying an expensive RAID card.
- Level of abstraction. Linux software RAID can distribute data across ATA, SCSI, iSCSI, SAN, network or any other block device. It is block device agnostic. Hardware RAID most likely cannot even span a single card.
- Hardware RAID has a field history of bad firmwares corrupting data, locking up, and otherwise behaving poorly under load. (certainly this is highly dependent on card model and firmware version)
- Hardware RAID firmwares have a very limited support lifetime. You cannot get firmware updates for older hardware. Sometimes the vendor even ceases to exist.
- Each hardware RAID has a different management interface, and level of feature support.
- Your hardware RAID featureset is largely locked in stone, at purchase time. With software RAID, the featureset grows with time, as new features are added to Linux... no hardware upgrade required.
- Additional RAID mode support. Most hardware controllers don't support RAID-6 as Linux software RAID does, and Linux will soon be adding RAID-5E and RAID-6E support.
- Error handling and logging varies from vendor to vendor (and card to card), with hardware RAID.
- Many ATA-based hardware RAID solutions either (a) fail to manage disk lifetimes via SMART, or (b) manage SMART diagnostics in a non-standard way.
Why prefer Linux hardware RAID?
- All the efficiencies that may be derived from reducing the number of copies of each WRITE (system -> controller, when compared to the software RAID case).
- Software RAID may saturate PCI bus bandwidth long before a hardware RAID card does (this presumes multiple devices on a single PCI bus).
- RAID5 XOR calculations are not performed on the host CPU, freeing the host CPU from other tasks and preventing host CPU saturation under load or DoS.
- It is easy to parallelize RAID operations, simply by purchasing more HW RAID cards, without affecting host CPU usage.
- Battery backup on high end cards allows faster journalled rebuilds.
- Battery-backed write-back cache may improve write throughput.
As an aside...
- Some cards export hardware RAID capabilities, such as XOR or RAID1 offload, but allow full OS control of the operations in lieu of a firmware. Linux does not support this programming model well.
FAQ
Responses to specific points often made in soft-vs-hardware RAID discussions.
- "Hardware RAID is always better."
No solution is always better than another solution. - "Software RAID is always better."
Ditto.