KfWiki: File System Striping

FileSystem » FileSystemStriping

File System Striping

This can be useful for building filesystem(s) for a database system. I will be using lvm to stripe data across disks. To enable stripping, one should have RAID as underlying storage to reduce the chances of loosing the entire volume because of a single disk failure.

Create physical volumes

Supposedly you have an underlying storage system, say RAID-10 on a hardware controller. Divide it into 3 equal parts and prep them for LVM use

pvcreate /dev/cciss/c0d1p1
pvcreate /dev/cciss/c0d1p2
pvcreate /dev/cciss/c0d1p3

Create volume groups

This is boring. If the volumes are not storing big files, a 4M extent should be sufficient

vgcreate -s4M vg0 /dev/cciss/c0d1p1 /dev/cciss/c0d1p2 /dev/cciss/c0d1p3

Create logical volumes

The interleave option here is important. This ensures IO are spread across the 3 PV's

lvcreate -i3 -I128K -L50G -ndata vg0
lvcreate -i3 -I128K -L5G -nlog vg0

Create filesystem

mkfs...

Benchmark

# bonnie+ + -d /mountpoint -r 512 -u root -s 2g -n0 -q -f -b

Without LVM striping

Version 1.03e       ------Sequential Output------ --Sequential Input- --Random-
					-Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP  /sec %CP
abox 2G           91039  24 82257  14           2203381 100 +++++ +++
abox,2G,,,91039,24,82257,14,,,2203381,100,+++++,+++,,,,,,,,,,,,,

With LVM striping
Write is 40% faster!

Version 1.03e       ------Sequential Output------ --Sequential Input- --Random-
					-Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP  /sec %CP
bbox 2G           126815  34 132786  23           2309730  99 +++++ +++
bbox,2G,,,126815,34,132786,23,,,2309730,99,+++++,+++,,,,,,,,,,,,,

3par

Version 1.03e       ------Sequential Output------ --Sequential Input- --Random-
					-Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP  /sec %CP
cbox  2G           316609  67 346639  50           2817900  99 +++++ +++
cbox,2G,,,316609,67,346639,50,,,2817900,99,+++++,+++,,,,,,,,,,,,,

3par w/ lvm stripping
strange, write thruput has reduced here

Version 1.03e       ------Sequential Output------ --Sequential Input- --Random-
					-Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP  /sec %CP
dbox       2G           179441  43 190579  30           2841138 100 +++++ +++
dbox,2G,,,179441,43,190579,30,,,2841138,100,+++++,+++,,,,,,,,,,,,,