■ファイルシステムの解除(drbd-1aのみ)
umount /mnt/drbd/zyyx
■スレーブへ降格(drbd-1aのみ)
drbdadm secondary all
■プライマリへ昇格(drbd-1bのみ)
drbdadm primary all
■ファイルシステムのマウント(drbd-1bのみ)
mkdir -p /mnt/drbd/zyyx
mount /dev/drbd0 /mnt/drbd/zyyx
■ちゃんと同期されているか確認(drbd-1bのみ)
# cat /mnt/drbd/zyyx/test.txt
this file is in drbd file system
■ベンチマークソフトの導入(共通)
yum -y install libaio-devel
cd /usr/src
wget http://brick.kernel.dk/snaps/fio-1.58.tar.gz
tar xzvf fio-1.58.tar.gz
cd fio-1.58
make
make install
■速度の計測
・ローカル環境
# fio -filename=/mnt/local1G -direct=1 -rw=randwrite -bs=1m -size=1G -numjobs=4 -runtime=10 -group_reporting -name=file1
file1: (g=0): rw=randwrite, bs=1M-1M/1M-1M, ioengine=sync, iodepth=1
...
file1: (g=0): rw=randwrite, bs=1M-1M/1M-1M, ioengine=sync, iodepth=1
fio 1.58
Starting 4 processes
file1: Laying out IO file(s) (1 file(s) / 1024MB)
Jobs: 4 (f=4): [wwww] [100.0% done] [0K/30408K /s] [0 /29 iops] [eta 00m:00s]
file1: (groupid=0, jobs=4): err= 0: pid=31670
write: io=319488KB, bw=31611KB/s, iops=30 , runt= 10107msec
clat (msec): min=29 , max=552 , avg=35.47, stdev=41.48
lat (msec): min=29 , max=552 , avg=35.47, stdev=41.48
bw (KB/s) : min= 1278, max=17520, per=24.77%, avg=7829.02, stdev=2868.20
cpu : usr=0.00%, sys=0.20%, ctx=714, majf=0, minf=99
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued r/w/d: total=0/312/0, short=0/0/0
lat (msec): 50=98.72%, 250=0.64%, 500=0.32%, 750=0.32%
Run status group 0 (all jobs):
WRITE: io=319488KB, aggrb=31610KB/s, minb=32369KB/s, maxb=32369KB/s, mint=10107msec, maxt=10107msec
Disk stats (read/write):
xvde1: ios=0/7425, merge=0/0, ticks=0/133277, in_queue=134281, util=98.70%
・同ゾーン
# fio -filename=/mnt/drbd/zyyx/remote1G -direct=1 -rw=randwrite -bs=1m -size=1G -numjobs=4 -runtime=10 -group_reporting -name=file1
file1: (g=0): rw=randwrite, bs=1M-1M/1M-1M, ioengine=sync, iodepth=1
...
file1: (g=0): rw=randwrite, bs=1M-1M/1M-1M, ioengine=sync, iodepth=1
fio 1.58
Starting 4 processes
file1: Laying out IO file(s) (1 file(s) / 1024MB)
Jobs: 4 (f=4): [wwww] [100.0% done] [0K/25140K /s] [0 /23 iops] [eta 00m:00s]
file1: (groupid=0, jobs=4): err= 0: pid=31697
write: io=183296KB, bw=17478KB/s, iops=17 , runt= 10487msec
clat (msec): min=36 , max=292 , avg=58.58, stdev=54.42
lat (msec): min=36 , max=292 , avg=58.58, stdev=54.42
bw (KB/s) : min= 97, max=25069, per=89.51%, avg=15644.40, stdev=8701.23
cpu : usr=0.02%, sys=0.17%, ctx=519, majf=0, minf=99
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued r/w/d: total=0/179/0, short=0/0/0
lat (msec): 50=87.71%, 100=2.79%, 250=7.26%, 500=2.23%
Run status group 0 (all jobs):
WRITE: io=183296KB, aggrb=17478KB/s, minb=17897KB/s, maxb=17897KB/s, mint=10487msec, maxt=10487msec
Disk stats (read/write):
drbd0: ios=0/4317, merge=0/0, ticks=0/139396, in_queue=145439, util=100.00%
・別ゾーン
# fio -filename=/mnt/drbd/zyyx/remote1G -direct=1 -rw=randw
rite -bs=1m -size=1G -numjobs=4 -runtime=10 -group_reporting -name=file1
file1: (g=0): rw=randwrite, bs=1M-1M/1M-1M, ioengine=sync, iodepth=1
...
file1: (g=0): rw=randwrite, bs=1M-1M/1M-1M, ioengine=sync, iodepth=1
fio 1.58
Starting 4 processes
file1: Laying out IO file(s) (1 file(s) / 1024MB)
Jobs: 4 (f=4): [wwww] [100.0% done] [0K/10485K /s] [0 /10 iops] [eta 00m:00s]
file1: (groupid=0, jobs=4): err= 0: pid=3221
write: io=181248KB, bw=17328KB/s, iops=16 , runt= 10460msec
clat (msec): min=40 , max=335 , avg=59.09, stdev=45.68
lat (msec): min=40 , max=335 , avg=59.09, stdev=45.68
bw (KB/s) : min= 97, max=22528, per=90.21%, avg=15630.00, stdev=8596.54
cpu : usr=0.01%, sys=0.13%, ctx=511, majf=0, minf=99
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued r/w/d: total=0/177/0, short=0/0/0
lat (msec): 50=73.45%, 100=20.34%, 250=3.39%, 500=2.82%
Run status group 0 (all jobs):
WRITE: io=181248KB, aggrb=17327KB/s, minb=17743KB/s, maxb=17743KB/s, mint=10460msec, maxt=10460msec
Disk stats (read/write):
drbd0: ios=0/4268, merge=0/0, ticks=0/155106, in_queue=161816, util=99.83%
同リージョンならゾーンが異なっても速度が変わらないようです。
ゾーンが違うと違うデータセンターに収容されるはずなので
別ゾーンで組んだ方が災害には強いのかもしれません。