Changes RSS

Testing Linux software raid configuration

Work in progress, these are rough notes…

Version 1.03d       ------Sequential Output------ --Sequential Input- --Random-
                    -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP  /sec %CP
filez            8G  5698  14  5692   5  4596   8 27752  63 70331  34 171.0   0
                    ------Sequential Create------ --------Random Create--------
                    -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
              files  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP
                 16   613  12 +++++ +++   441   8   582  12 +++++ +++   410   8
filez,8G,5698,14,5692,5,4596,8,27752,63,70331,34,171.0,0,16,613,12,+++++,+++,441,8,582,12,+++++,+++,410,8

En Hitachi-disk på Adaptec-kontroller, satt opp som Volume med Write-cache på kontroller.
En partisjon på disk, 16GB..
Filsystemet er EXT3, opprettet med
mkfs.ext3 -b 4096 -i 4096 -m 0 /dev/sdb1

Version 1.03d       ------Sequential Output------ --Sequential Input- --Random- 
                    -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks-- 
Machine        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP  /sec %CP 
filez            8G  7375  35  7442  24  5718  11 28081  71 63743  45 219.4   1 
                    ------Sequential Create------ --------Random Create-------- 
                    -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete-- 
              files  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP 
                 16  5968  97 +++++ +++  7104  99  6789  93 +++++ +++  8057  99 
filez,8G,7375,35,7442,24,5718,11,28081,71,63743,45,219.4,1,16,5968,97,+++++,+++,7104,99,6789,93,+++++,+++,8057,99   

An Hitachi-disk på Asus-onbard AHCIkontroller, samme partisjonsoppsett som over:

Version 1.03d       ------Sequential Output------ --Sequential Input- --Random-
                    -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP  /sec %CP
filez            8G 21308  81 30172  69 37773  48 39839  94 127851  87 261.0   2
                    ------Sequential Create------ --------Random Create--------
                    -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
              files  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP
                 16  6175  96 +++++ +++  6875  99  6974  96 +++++ +++  7023  99
filez,8G,21308,81,30172,69,37773,48,39839,94,127851,87,261.0,2,16,6175,96,+++++,+++,6875,99,6974,96,+++++,+++,7023,99

SÅ, over SATA2 skal hver disk kunne levere 20(29)/40(125) i det minste...

mdadm \
--create --verbose /dev/md0 \
 --level=raid5 \
 --metadata=1 \
 --raid-devices=4 \
    /dev/sdb \
    /dev/sdc \
    /dev/sdd \
    /dev/sde \
 --chunk=256

pvcreate /dev/md0

vgcreate -s 32M raid5v /dev/md0

lvcreate -n test -L 32G raid5v

mkfs.ext3 -b 4096 -E stride=64 -i 4096 -m 0 /dev/raid5v/test

Version 1.03d       ------Sequential Output------ --Sequential Input- --Random-
                    -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP  /sec %CP
filez            8G  9513  49 14860  50 16499  40 22149  74 31935  41 300.6   3
                    ------Sequential Create------ --------Random Create--------
                    -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
              files  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP
                 16  3885  98 +++++ +++  4938  91  2157  53 +++++ +++  2358  43
filez,8G,9513,49,14860,50,16499,40,22149,74,31935,41,300.6,3,16,3885,98,+++++,+++,4938,91,2157,53,+++++,+++,2358,43

Samme RAID5-sett rebuild/init var ferdig:

Version 1.03d       ------Sequential Output------ --Sequential Input- --Random-
                    -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP  /sec %CP
filez            8G 14559  69 20433  60 20453  50 26155  92 50916  87 441.5   5
                    ------Sequential Create------ --------Random Create--------
                    -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
              files  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP
                 16  3880  96 +++++ +++  5074  91  3918  98 +++++ +++  5181  94
filez,8G,14559,69,20433,60,20453,50,26155,92,50916,87,441.5,5,16,3880,96,+++++,+++,5074,91,3918,98,+++++,+++,5181,94


Samme RAID5-sett, men mkfs.ext3 uten -E stride=64 opsjonen
Version 1.03d       ------Sequential Output------ --Sequential Input- --Random-
                    -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP  /sec %CP
filez            8G 13725  67 17437  59 19201  49 26382  92 52236  86 455.7   4
                    ------Sequential Create------ --------Random Create--------
                    -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
              files  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP
                 16  3916  94 +++++ +++  4963  91  4877  97 +++++ +++  5486  92
filez,8G,13725,67,17437,59,19201,49,26382,92,52236,86,455.7,4,16,3916,94,+++++,+++,4963,91,4877,97,+++++,+++,5486,92

mdadm \
--create --verbose /dev/md0 \
 --level=raid5 \
 --metadata=1 \
 --raid-devices=4 \
    /dev/sdb \
    /dev/sdc \
    /dev/sdd \
    /dev/sde \
 --chunk=128

pvcreate /dev/md0

vgcreate -s 32M raid5v /dev/md0

lvcreate -n test -L 32G raid5v

mkfs.ext3 -b 4096 -E stride=32 -i 4096 -m 0 /dev/raid5v/test

under build

Version 1.03d       ------Sequential Output------ --Sequential Input- --Random-
                    -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP  /sec %CP
filez            8G  7881  34 11826  31 17959  28 22926  61 33085  24 153.5   1
                    ------Sequential Create------ --------Random Create--------
                    -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
              files  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP
                 16  1528  27 +++++ +++  1783  23  1751  27 +++++ +++  2629  36
filez,8G,7881,34,11826,31,17959,28,22926,61,33085,24,153.5,1,16,1528,27,+++++,+++,1783,23,1751,27,+++++,+++,2629,36

Etter build

Version 1.03d       ------Sequential Output------ --Sequential Input- --Random-
                    -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP  /sec %CP
filez            8G 16305  68 21675  55 23201  44 31438  92 78377  86 447.8   4
                    ------Sequential Create------ --------Random Create--------
                    -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
              files  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP
                 16  7215  97 +++++ +++  7768  99  8037  93 +++++ +++  8177  99
filez,8G,16305,68,21675,55,23201,44,31438,92,78377,86,447.8,4,16,7215,97,+++++,+++,7768,99,8037,93,+++++,+++,8177,99

mdadm \
--create --verbose /dev/md0 \
 --level=raid5 \
 --metadata=1 \
 --raid-devices=4 \
    /dev/sdb \
    /dev/sdc \
    /dev/sdd \
    /dev/sde \
 --chunk=64

pvcreate /dev/md0

vgcreate -s 32M raid5v /dev/md0

lvcreate -n test -L 32G raid5v

mkfs.ext3 -b 4096 -E stride=16 -i 4096 -m 0 /dev/raid5v/test

Under build:
Version 1.03d       ------Sequential Output------ --Sequential Input- --Random-
                    -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP  /sec %CP
filez            8G  7009  33 11222  32 14150  16 16201  47 21514  28 164.9   2
                    ------Sequential Create------ --------Random Create--------
                    -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
              files  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP
                 16  1543  20 28677  42  1705  24  2142  26 27715  16  2317  29
filez,8G,7009,33,11222,32,14150,16,16201,47,21514,28,164.9,2,16,1543,20,28677,42,1705,24,2142,26,27715,16,2317,29

Etter build-complete
Version 1.03d       ------Sequential Output------ --Sequential Input- --Random-
                    -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP  /sec %CP
filez            8G 16100  66 21336  54 22765  45 28652  93 61033  89 440.1   4
                    ------Sequential Create------ --------Random Create--------
                    -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
              files  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP
                 16  7692  98 +++++ +++  7406  98  7324  92 +++++ +++  7893  98
filez,8G,16100,66,21336,54,22765,45,28652,93,61033,89,440.1,4,16,7692,98,+++++,+++,7406,98,7324,92,+++++,+++,7893,98


echo 8192 > /sys/block/md0/md/stripe_cache_size 
Status for cache skal kunne leses fra /sys/block/md0/md/stripe_cache_active
Version 1.03d       ------Sequential Output------ --Sequential Input- --Random-
                    -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP  /sec %CP
filez            8G 15685  64 21053  54 21318  44 27955  91 58725  87 436.2   4
                    ------Sequential Create------ --------Random Create--------
                    -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
              files  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP
                 16  7190  97 +++++ +++  7234  96  7140  92 +++++ +++  7928  95
filez,8G,15685,64,21053,54,21318,44,27955,91,58725,87,436.2,4,16,7190,97,+++++,+++,7234,96,7140,92,+++++,+++,7928,95

echo 512 > /sys/block/md0/md/stripe_cache_size 

Version 1.03d       ------Sequential Output------ --Sequential Input- --Random-
                    -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP  /sec %CP
filez            8G 15716  65 21201  53 22391  46 29170  91 64344  87 442.4   5
                    ------Sequential Create------ --------Random Create--------
                    -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
              files  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP
                 16  7258  96 +++++ +++  7224  91  7335  97 +++++ +++  7691  91
filez,8G,15716,65,21201,53,22391,46,29170,91,64344,87,442.4,5,16,7258,96,+++++,+++,7224,91,7335,97,+++++,+++,7691,91