Coder Social home page Coder Social logo

iobench.sh's Introduction

sysbench fileio based disk io benchmark with common metrics

Just another simple wrapper over sysbench fileio to output a table of common metrics.

Usage: iobench.sh [-t] <fileNum>/<fileTotalSize>

sysbench fileio will create some files under the current working directory named starts from test_file.0 to test_file.<fileNum> and have a total sum of sizes equal to <fileTotalSize>, so the size of each test_file is <fileNum>/<fileTotalSize>.

iobench.sh will invoke sysbench fileio to do combinations of some common params and convert the human-readable report in its stdout to space separated tuples into stderr for any further summary analysis about the performance of the file system and its block device behind testing pwd.

Example:

$ ./iobench.sh 2/1G 2> >((sed -u 1q; sort -k2.5,2.6 -k1h,1 -k3,3) | column -t)
testing the file system /dev/sdb1 behind /mnt
prepare 2 files with a total size of 1G
sysbench 1.0.20 (using system LuaJIT 2.1.0-beta3)

2 files, 524288Kb each, 1024Mb total
Creating files for the test...
Extra file open flags: (none)
Creating file test_file.0
Creating file test_file.1
1073741824 bytes written in 8.93 seconds (114.70 MiB/sec).
running rndrd with block size 4k and 1 threads
running rndwr with block size 4k and 1 threads
running seqrd with block size 4k and 1 threads
running seqwr with block size 4k and 1 threads
prepare 2 files with a total size of 1G
sysbench 1.0.20 (using system LuaJIT 2.1.0-beta3)

2 files, 524288Kb each, 1024Mb total
Creating files for the test...
Extra file open flags: (none)
Reusing existing file test_file.0
Reusing existing file test_file.1
No bytes written.
running rndrw with block size 4k and 1 threads
running rndrd with block size 1m and 1 threads
running rndwr with block size 1m and 1 threads
running seqrd with block size 1m and 1 threads
running seqwr with block size 1m and 1 threads
prepare 2 files with a total size of 1G
sysbench 1.0.20 (using system LuaJIT 2.1.0-beta3)

2 files, 524288Kb each, 1024Mb total
Creating files for the test...
Extra file open flags: (none)
Reusing existing file test_file.0
Reusing existing file test_file.1
No bytes written.
running rndrw with block size 1m and 1 threads
running rndrd with block size 4k and 16 threads
running rndwr with block size 4k and 16 threads
running seqrd with block size 4k and 16 threads
running seqwr with block size 4k and 16 threads
prepare 2 files with a total size of 1G
sysbench 1.0.20 (using system LuaJIT 2.1.0-beta3)

2 files, 524288Kb each, 1024Mb total
Creating files for the test...
Extra file open flags: (none)
Reusing existing file test_file.0
Reusing existing file test_file.1
No bytes written.
running rndrw with block size 4k and 16 threads
running rndrd with block size 1m and 16 threads
running rndwr with block size 1m and 16 threads
running seqrd with block size 1m and 16 threads
running seqwr with block size 1m and 16 threads
prepare 2 files with a total size of 1G
sysbench 1.0.20 (using system LuaJIT 2.1.0-beta3)

2 files, 524288Kb each, 1024Mb total
Creating files for the test...
Extra file open flags: (none)
Reusing existing file test_file.0
Reusing existing file test_file.1
No bytes written.
running rndrw with block size 1m and 16 threads
sysbench 1.0.20 (using system LuaJIT 2.1.0-beta3)

Removing test files...
blockSize  testMode  threads  rdIOps      wrIOps    rdMiBps   wrMiBps  latMsAvg  latMs95th
4k         rndrd     1        1008748.32  0.00      3940.42   0.00     0.00      0.00
4k         seqrd     1        1431434.07  0.00      5591.54   0.00     0.00      0.00
4k         rndrd     16       1352502.73  0.00      5283.21   0.00     0.01      0.00
4k         seqrd     16       1704782.16  0.00      6659.31   0.00     0.01      0.00
64k        rndrd     1        118493.54   0.00      7405.85   0.00     0.01      0.01
64k        seqrd     1        160453.34   0.00      10028.33  0.00     0.01      0.01
64k        rndrd     16       248927.40   0.00      15557.96  0.00     0.06      0.01
64k        seqrd     16       264553.16   0.00      16534.57  0.00     0.06      0.01
4k         rndrw     1        13615.73    9077.15   53.19     35.46    0.04      0.00
4k         rndrw     16       13704.99    9136.66   53.54     35.69    0.69      0.00
64k        rndrw     1        4659.53     3106.35   291.22    194.15   0.13      0.02
64k        rndrw     16       4668.42     3112.28   291.78    194.52   2.02      0.03
4k         rndwr     1        0.00        9090.29   0.00      35.51    0.11      0.00
4k         seqwr     1        0.00        49497.10  0.00      193.35   0.02      0.00
4k         rndwr     16       0.00        9178.70   0.00      35.85    1.71      0.00
4k         seqwr     16       0.00        43639.89  0.00      170.47   0.36      0.01
64k        rndwr     1        0.00        3179.90   0.00      198.74   0.31      0.03
64k        seqwr     1        0.00        3189.06   0.00      199.32   0.31      0.04
64k        rndwr     16       0.00        3232.92   0.00      202.06   4.85      0.03
64k        seqwr     16       0.00        3201.00   0.00      200.06   4.90      0.09

iobench.sh's People

Contributors

n0099 avatar

Stargazers

 avatar

Watchers

 avatar

iobench.sh's Issues

基于`sysbench fileio`对 Azure 上的`Ddsv4` `Ddsv5` `Dadsv5` vm系列中的`D2`大小+`P10 Premium SSD`的benchmark

cd ~
git clone https://github.com/n0099/iobench.sh
~/iobench.sh/iobench.sh 2/1G 2> >((sed -u 1q; sort -k2.5,2.6 -k3,3 -k1hr,1) | column -t > sda)
cd /mnt
sudo chmod a+w .
~/iobench.sh/iobench.sh 2/1G 2> >((sed -u 1q; sort -k2.5,2.6 -k3,3 -k1hr,1) | column -t > ~/sdb)

iobench.zip

sdaP10硬盘,IO 需要经过机房内网,但开启了 host level rw cache 所以缓存内的读吞吐量远超P10本身的上限
sdb是 vm 附带的 temp disk,IO 不需要经过机房内网
.busysda是在同时跑着 mysql 等 IO 密集业务负荷时进行的,所以反应了既有业务下的突发需求性能并且容易让 host level cache 失效从而反应其内网的真实性能
不带后缀的都是在关闭业务无压力时进行的

sda.D2dsv4.busy

blockSize testMode threads rdIOps wrIOps rdMiBps wrMiBps latMsAvg latMs95th
4k rndrd 1 39706.06 0.00 155.10 0.00 0.02 0.16
4k seqrd 1 185742.36 0.00 725.56 0.00 0.01 0.00
1m rndrd 1 5545.57 0.00 5545.57 0.00 0.18 0.23
1m seqrd 1 2804.28 0.00 2804.28 0.00 0.36 0.92
4k rndrd 16 691118.08 0.00 2699.68 0.00 0.02 0.00
4k seqrd 16 786871.75 0.00 3073.72 0.00 0.02 0.00
1m rndrd 16 6230.92 0.00 6230.92 0.00 2.55 24.38
1m seqrd 16 6381.08 0.00 6381.08 0.00 2.50 24.38
4k rndrw 1 4929.71 3286.47 19.26 12.84 0.12 0.01
1m rndrw 1 130.89 87.26 130.89 87.26 4.49 0.48
4k rndrw 16 4578.38 3052.25 17.88 11.92 2.05 0.02
1m rndrw 16 132.46 88.31 132.46 88.31 70.85 30.26
4k rndwr 1 0.00 3316.28 0.00 12.95 0.29 0.01
4k seqwr 1 0.00 13786.94 0.00 53.86 0.07 0.01
1m rndwr 1 0.00 78.57 0.00 78.57 12.47 0.46
1m seqwr 1 0.00 69.48 0.00 69.48 14.10 1.18
4k rndwr 16 0.00 2361.81 0.00 9.23 6.62 0.02
4k seqwr 16 0.00 29827.44 0.00 116.51 0.52 0.02
1m rndwr 16 0.00 103.97 0.00 103.97 135.75 132.49
1m seqwr 16 0.00 87.60 0.00 87.60 164.51 257.95

sda.D2dsv4

blockSize testMode threads rdIOps wrIOps rdMiBps wrMiBps latMsAvg latMs95th
4k rndrd 1 480576.60 0.00 1877.25 0.00 0.00 0.00
4k seqrd 1 587713.43 0.00 2295.76 0.00 0.00 0.00
1m rndrd 1 4125.15 0.00 4125.15 0.00 0.24 0.28
1m seqrd 1 4047.72 0.00 4047.72 0.00 0.25 0.30
4k rndrd 16 645639.07 0.00 2522.03 0.00 0.02 0.00
4k seqrd 16 745762.54 0.00 2913.13 0.00 0.02 0.00
1m rndrd 16 4994.71 0.00 4994.71 0.00 3.18 28.67
1m seqrd 16 5063.10 0.00 5063.10 0.00 3.15 28.67
4k rndrw 1 5050.01 3366.67 19.73 13.15 0.12 0.01
1m rndrw 1 156.62 104.41 156.62 104.41 3.68 0.56
4k rndrw 16 5200.88 3467.65 20.32 13.55 1.80 0.03
1m rndrw 16 124.52 83.01 124.52 83.01 72.02 16.71
4k rndwr 1 0.00 3474.19 0.00 13.57 0.28 0.01
4k seqwr 1 0.00 14011.66 0.00 54.73 0.07 0.01
1m rndwr 1 0.00 79.95 0.00 79.95 12.26 0.61
1m seqwr 1 0.00 69.21 0.00 69.21 14.16 1.30
4k rndwr 16 0.00 2651.09 0.00 10.36 5.91 0.02
4k seqwr 16 0.00 29770.67 0.00 116.29 0.53 0.02
1m rndwr 16 0.00 101.35 0.00 101.35 141.57 38.94
1m seqwr 16 0.00 83.67 0.00 83.67 171.03 44.98

sda.D2dsv5

blockSize testMode threads rdIOps wrIOps rdMiBps wrMiBps latMsAvg latMs95th
4k rndrd 1 1006153.83 0.00 3930.29 0.00 0.00 0.00
4k seqrd 1 1225677.28 0.00 4787.80 0.00 0.00 0.00
1m rndrd 1 7842.28 0.00 7842.28 0.00 0.13 0.14
1m seqrd 1 7725.69 0.00 7725.69 0.00 0.13 0.14
4k rndrd 16 1468564.65 0.00 5736.58 0.00 0.01 0.00
4k seqrd 16 1649069.92 0.00 6441.68 0.00 0.01 0.00
1m rndrd 16 8883.60 0.00 8883.60 0.00 1.79 24.38
1m seqrd 16 8877.75 0.00 8877.75 0.00 1.79 24.38
4k rndrw 1 4755.52 3170.35 18.58 12.38 0.12 0.00
1m rndrw 1 253.03 168.69 253.03 168.69 2.29 0.24
4k rndrw 16 4259.05 2839.37 16.64 11.09 2.21 0.01
1m rndrw 16 171.08 114.05 171.08 114.05 54.85 8.28
4k rndwr 1 0.00 3470.88 0.00 13.56 0.28 0.00
4k seqwr 1 0.00 13643.30 0.00 53.29 0.07 0.00
1m rndwr 1 0.00 166.36 0.00 166.36 5.89 0.26
1m seqwr 1 0.00 103.51 0.00 103.51 9.36 0.54
4k rndwr 16 0.00 2326.47 0.00 9.09 6.73 0.01
4k seqwr 16 0.00 41758.53 0.00 163.12 0.38 0.01
1m rndwr 16 0.00 123.94 0.00 123.94 122.65 20.37
1m seqwr 16 0.00 102.55 0.00 102.55 126.31 36.24

sda.D2adsv5

blockSize testMode threads rdIOps wrIOps rdMiBps wrMiBps latMsAvg latMs95th
4k rndrd 1 1072086.42 0.00 4187.84 0.00 0.00 0.00
4k seqrd 1 1484819.50 0.00 5800.08 0.00 0.00 0.00
1m rndrd 1 14057.86 0.00 14057.86 0.00 0.07 0.09
1m seqrd 1 14108.59 0.00 14108.59 0.00 0.07 0.09
4k rndrd 16 1629370.86 0.00 6364.73 0.00 0.01 0.00
4k seqrd 16 1940870.87 0.00 7581.53 0.00 0.01 0.00
1m rndrd 16 21031.89 0.00 21031.89 0.00 0.76 0.13
1m seqrd 16 20914.38 0.00 20914.38 0.00 0.76 0.13
4k rndrw 1 5067.65 3378.43 19.80 13.20 0.12 0.00
1m rndrw 1 181.35 120.90 181.35 120.90 3.21 0.20
4k rndrw 16 4566.25 3044.17 17.84 11.89 2.06 0.01
1m rndrw 16 168.69 112.46 168.69 112.46 54.62 4.91
4k rndwr 1 0.00 3425.41 0.00 13.38 0.29 0.00
4k seqwr 1 0.00 25583.41 0.00 99.94 0.04 0.00
1m rndwr 1 0.00 103.56 0.00 103.56 8.97 0.20
1m seqwr 1 0.00 89.63 0.00 89.63 10.94 0.56
4k rndwr 16 0.00 2371.91 0.00 9.27 6.57 0.01
4k seqwr 16 0.00 41617.10 0.00 162.57 0.38 0.01
1m rndwr 16 0.00 125.74 0.00 125.74 118.20 16.41
1m seqwr 16 0.00 97.53 0.00 97.53 134.64 44.98

sdb.D2dsv4.busy

blockSize testMode threads rdIOps wrIOps rdMiBps wrMiBps latMsAvg latMs95th
4k rndrd 1 534324.08 0.00 2087.20 0.00 0.00 0.00
4k seqrd 1 689408.74 0.00 2693.00 0.00 0.00 0.00
1m rndrd 1 5525.08 0.00 5525.08 0.00 0.18 0.22
1m seqrd 1 5912.00 0.00 5912.00 0.00 0.17 0.20
4k rndrd 16 515271.32 0.00 2012.78 0.00 0.03 0.00
4k seqrd 16 894866.27 0.00 3495.57 0.00 0.02 0.00
1m rndrd 16 6728.51 0.00 6728.51 0.00 2.36 23.95
1m seqrd 16 6013.90 0.00 6013.90 0.00 2.65 24.38
4k rndrw 1 28345.71 18897.14 110.73 73.82 0.02 0.01
1m rndrw 1 179.65 119.76 179.65 119.76 3.27 0.38
4k rndrw 16 28891.83 19261.22 112.86 75.24 0.33 0.01
1m rndrw 16 186.84 124.56 186.84 124.56 50.31 20.74
4k rndwr 1 0.00 19210.84 0.00 75.04 0.05 0.00
4k seqwr 1 0.00 28544.05 0.00 111.50 0.03 0.01
1m rndwr 1 0.00 122.52 0.00 122.52 7.71 0.44
1m seqwr 1 0.00 111.48 0.00 111.48 8.80 0.94
4k rndwr 16 0.00 19370.95 0.00 75.67 0.81 0.01
4k seqwr 16 0.00 29467.31 0.00 115.11 0.53 0.02
1m rndwr 16 0.00 136.90 0.00 136.90 110.97 71.83
1m seqwr 16 0.00 117.21 0.00 117.21 113.98 383.33

sdb.D2dsv4

blockSize testMode threads rdIOps wrIOps rdMiBps wrMiBps latMsAvg latMs95th
4k rndrd 1 459889.58 0.00 1796.44 0.00 0.00 0.00
4k seqrd 1 605828.66 0.00 2366.52 0.00 0.00 0.00
1m rndrd 1 4066.44 0.00 4066.44 0.00 0.25 0.29
1m seqrd 1 4091.61 0.00 4091.61 0.00 0.24 0.29
4k rndrd 16 651377.29 0.00 2544.44 0.00 0.02 0.00
4k seqrd 16 715851.81 0.00 2796.30 0.00 0.02 0.00
1m rndrd 16 4816.32 0.00 4816.32 0.00 3.31 28.67
1m seqrd 16 4774.60 0.00 4774.60 0.00 3.34 28.67
4k rndrw 1 28591.14 19060.76 111.68 74.46 0.02 0.01
1m rndrw 1 178.61 119.07 178.61 119.07 3.29 0.57
4k rndrw 16 28932.06 19288.04 113.02 75.34 0.32 0.01
1m rndrw 16 186.05 124.03 186.05 124.03 50.21 23.10
4k rndwr 1 0.00 19132.37 0.00 74.74 0.05 0.01
4k seqwr 1 0.00 28545.83 0.00 111.51 0.03 0.01
1m rndwr 1 0.00 121.62 0.00 121.62 7.76 0.64
1m seqwr 1 0.00 109.66 0.00 109.66 8.94 1.25
4k rndwr 16 0.00 19349.69 0.00 75.58 0.81 0.01
4k seqwr 16 0.00 29254.25 0.00 114.27 0.54 0.02
1m rndwr 16 0.00 145.76 0.00 145.76 107.39 41.85
1m seqwr 16 0.00 117.12 0.00 117.12 125.80 55.82

sdb.D2dsv5

blockSize testMode threads rdIOps wrIOps rdMiBps wrMiBps latMsAvg latMs95th
4k rndrd 1 1026993.19 0.00 4011.69 0.00 0.00 0.00
4k seqrd 1 1232872.75 0.00 4815.91 0.00 0.00 0.00
1m rndrd 1 7785.79 0.00 7785.79 0.00 0.13 0.14
1m seqrd 1 7705.53 0.00 7705.53 0.00 0.13 0.14
4k rndrd 16 1496480.68 0.00 5845.63 0.00 0.01 0.00
4k seqrd 16 1729444.62 0.00 6755.64 0.00 0.01 0.00
1m rndrd 16 8905.27 0.00 8905.27 0.00 1.79 24.38
1m seqrd 16 8893.76 0.00 8893.76 0.00 1.79 24.38
4k rndrw 1 13465.82 8977.22 52.60 35.07 0.04 0.00
1m rndrw 1 297.77 198.51 297.77 198.51 1.97 0.24
4k rndrw 16 13743.56 9162.37 53.69 35.79 0.68 0.00
1m rndrw 16 307.52 205.18 307.52 205.18 30.56 11.87
4k rndwr 1 0.00 9076.85 0.00 35.46 0.11 0.00
4k seqwr 1 0.00 49423.34 0.00 193.06 0.02 0.00
1m rndwr 1 0.00 209.25 0.00 209.25 4.68 0.27
1m seqwr 1 0.00 199.29 0.00 199.29 4.92 0.52
4k rndwr 16 0.00 9204.40 0.00 35.95 1.70 0.00
4k seqwr 16 0.00 50670.03 0.00 197.93 0.31 0.01
1m rndwr 16 0.00 245.13 0.00 245.13 61.68 21.11
1m seqwr 16 0.00 197.64 0.00 197.64 72.47 39.65

sdb.D2adsv5

blockSize testMode threads rdIOps wrIOps rdMiBps wrMiBps latMsAvg latMs95th
4k rndrd 1 1107399.28 0.00 4325.78 0.00 0.00 0.00
4k seqrd 1 1490365.50 0.00 5821.74 0.00 0.00 0.00
1m rndrd 1 13778.88 0.00 13778.88 0.00 0.07 0.09
1m seqrd 1 13556.32 0.00 13556.32 0.00 0.07 0.10
4k rndrd 16 1654259.58 0.00 6461.95 0.00 0.01 0.00
4k seqrd 16 2012327.87 0.00 7860.66 0.00 0.01 0.00
1m rndrd 16 20505.81 0.00 20505.81 0.00 0.78 0.13
1m seqrd 16 20393.08 0.00 20393.08 0.00 0.78 0.13
4k rndrw 1 13618.73 9079.15 53.20 35.47 0.04 0.00
1m rndrw 1 297.55 198.37 297.55 198.37 1.98 0.20
4k rndrw 16 13698.83 9132.56 53.51 35.67 0.69 0.00
1m rndrw 16 309.71 206.62 309.71 206.62 30.33 1.50
4k rndwr 1 0.00 9099.64 0.00 35.55 0.11 0.00
4k seqwr 1 0.00 49601.52 0.00 193.76 0.02 0.00
1m rndwr 1 0.00 207.03 0.00 207.03 4.73 0.21
1m seqwr 1 0.00 199.15 0.00 199.15 4.92 0.45
4k rndwr 16 0.00 9197.73 0.00 35.93 1.70 0.00
4k seqwr 16 0.00 50377.79 0.00 196.79 0.31 0.01
1m rndwr 16 0.00 239.45 0.00 239.45 64.09 17.63
1m seqwr 16 0.00 197.92 0.00 197.92 70.71 30.26

ref

https://learn.microsoft.com/en-us/azure/virtual-machines/ddv4-ddsv4-series
https://learn.microsoft.com/en-us/azure/virtual-machines/ddv5-ddsv5-series
https://learn.microsoft.com/en-us/azure/virtual-machines/dasv5-dadsv5-series
https://learn.microsoft.com/en-us/azure/virtual-machines/disks-types#premium-ssds
https://learn.microsoft.com/en-us/azure/virtual-machines/premium-storage-performance
https://learn.microsoft.com/en-us/azure/virtual-machines/disks-performance
https://learn.microsoft.com/en-us/azure/virtual-machines/disks-benchmarks

https://azureprice.net/compare?vms=Standard_D2ds_v4%2CStandard_D2ds_v5
https://azureprice.net/compare?vms=Standard_D2ds_v4%2CStandard_D2ads_v5
https://azureprice.net/compare?vms=Standard_D2ads_v5%2CStandard_D2ds_v5

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.