Linux software raid

Create new raid

Create the partition tables on one disk (here sda) and copy the disk layout to the second one (here sdb)
(assuming you have two identical SATA-Disks: sda and sdb)

$ sfdisk -d /dev/sda | sfdisk /dev/sdb

Create a new software raid 1

mdadm --create --verbose /dev/md2 --level=1 --raid-devices=2 /dev/sda5 /dev/sdb5

If you want to create serveral raid devices at once try this:

for i in 6 7 8; do mdadm --create --verbose /dev/md$i --level=1 --raid-devices=2 /dev/sda$i /dev/sdb$i; done

Saving your setup to config file

Once you set up your array it would be helpful (but not necessary) to create a /etc/mdadm/mdadm.conf

mdadm --detail --scan > /etc/mdadm/mdadm.conf

you will get something like this:

ARRAY /dev/md1 level=raid1 num-devices=2 UUID=727232b4:09441cd2:2c40d2be:4d07af58
   devices=/dev/sda1,/dev/sdb1
ARRAY /dev/md5 level=raid1 num-devices=2 UUID=aa0be2f1:35f327ba:8231611c:30c1cd8d
   devices=/dev/sda5,/dev/sdb5
ARRAY /dev/md6 level=raid1 num-devices=2 UUID=9f9978f4:d7475f57:c42e1ed2:45b4665c
   devices=/dev/sda6,/dev/sdb6
ARRAY /dev/md7 level=raid1 num-devices=2 UUID=1fd86d6c:e32f550b:870fec6e:7f3724f5
   devices=/dev/sda7,/dev/sdb7
ARRAY /dev/md8 level=raid1 num-devices=2 UUID=a90171e7:98e53b47:0a4aa1b7:3561e627
   devices=/dev/sda8,/dev/sdb8

Every device used within your array has to be specified like this:

DEVICE /dev/sda* /dev/sdb*

Working with your raid

Stop an unmounted raid

mdadm --stop /dev/md2

Show details:

mdadm --detail /dev/md0

Output:

/dev/md0:
        Version : 00.90.03
  Creation Time : Tue Nov 18 08:41:45 2008
     Raid Level : raid1
     Array Size : 128384 (125.40 MiB 131.47 MB)
    Device Size : 128384 (125.40 MiB 131.47 MB)
   Raid Devices : 2
  Total Devices : 2
Preferred Minor : 0
    Persistence : Superblock is persistent

    Update Time : Tue Nov 18 14:56:49 2008
          State : clean
 Active Devices : 2
Working Devices : 2
 Failed Devices : 0
  Spare Devices : 0

           UUID : 4b2dce7e:0343afb2:26449295:6e62cb31
         Events : 0.4

    Number   Major   Minor   RaidDevice State
       0       8        1        0      active sync   /dev/sda1
       1       8       17        1      active sync   /dev/sdb1

Status in /proc/mdstat
You can see the progress of a (re)sync of you raid with the following command:

cat /proc/mdstat
#or
watch -n 3 cat /proc/mdstat

Output:

Personalities : [raid1]
md1 : active raid1 sda3[0] sdb3[1]
      158754240 blocks [2/2] [UU]

md0 : active raid1 sda1[0] sdb1[1]
      128384 blocks [2/2] [UU]

unused devices: <none>

Replace dead harddrive and resync your raid

Assumption: one of your drives died (raid 1), you replaced the drive while your server was powerd off. Now your server is running in degraded mode and the new drive is empty and waiting to be integrated into your raid:

Copy the partition table layout to the new drive (be sure, to specify the correct drives or you will end up in desaster!)

  • /dev/sda - good drive with your data
  • /dev/sdb - new drive, completely empty

Let's have a look at the raid arry:

 mdadm --detail --scan
ARRAY /dev/md1 level=raid1 num-devices=2
   devices=/dev/sda1
ARRAY /dev/md5 level=raid1 num-devices=2
   devices=/dev/sda5
ARRAY /dev/md6 level=raid1 num-devices=2
   devices=/dev/sda6
ARRAY /dev/md7 level=raid1 num-devices=2
   devices=/dev/sda7
ARRAY /dev/md8 level=raid1 num-devices=2
   devices=/dev/sda8

OK, /dev/sdb is missing. We copy the partition table from sda to sdb:

sfdisk -d /dev/sda | sfdisk /dev/sdb

And now we tell the raid array to sync the new partitions of sdb into the array:

for i in 1 5 6 7 8; do mdadm --add --verbose /dev/md$i /dev/sda$i /dev/sdb$i; done

Now the raid will sync the data back to sdb, where it belongs.

Repair your raid

If one of your raid sets run out of sync you can try the following.

As you can see, there is only one mirror online. sdb2 has been kicked of the array.

md0 : active raid1 sda2[0] 
      96320 blocks [2/2] [U_]

Well, what shall we do?

Ensure sdb2 has really been kicked off the array:

mdadm /dev/md0 –-fail /dev/sdb2 -–remove /dev/sdb2

Then re-add sdb2 to the array:

mdadm /dev/md0 –-add /dev/sdb2

This will start the initialisation of the raid. If there is no hardware defect we should soon be in business again.

Grow your raid

To grow your raid by adding an additional harddrive, see https://raid.wiki.kernel.org/index.php/Growing

Don't do this without a backup of your data!

The steps in short (growing a 3 disk raid5 to a 4 disk raid5):

# grow the raid
mdadm --add /dev/md0 /dev/sdd3
mdadm --grow /dev/md0 --raid-devices=4
 
# modify /etc/mdadm.conf to reflect the new layout (num-devices=4)
vim /etc/mdadm.conf
 
# if you use LVM on top of raid, grow the physical volume:
pvdisplay # to see the situation before
pvresize /dev/md0
pvdisplay # did we grow?
 
# grow the lv
lvextend -L +50G -n your_lv volume_group
 
# finally grow filesystem
# check the filesystem first
fsck /dev/mapper/volume_group-your_lv
# grow ext3 filesystem:
resize2fs /dev/mapper/volume_group-your_lv
linux/software-raid.txt · Last modified: 2010/10/26 09:48 by tb
CC Attribution-Noncommercial-Share Alike 3.0 Unported
www.chimeric.de Valid CSS Driven by DokuWiki do yourself a favour and use a real browser - get firefox!! Recent changes RSS feed Valid XHTML 1.0