[Gluster-users] small write speed problem on EBS, distributed replica

karol skocik karol.skocik at gmail.com
Mon Mar 21 08:27:51 UTC 2011

  I am in the process of evaluation of Gluster for major BI company,
but I was surprised by very small write performance on Amazon EBS.
Our setup is Gluster 3.1.2, distributed replica 2x2 on 64-bit m1.large
instances. Every server node has 1 EBS volume attached to it.
The configuration of the distributed replica is a default one, my
small attemps to improve performance (io-threads, disabled io-stats
and latency-measurement):

volume EBSVolume-posix
    type storage/posix
    option directory /mnt/ebs

volume EBSVolume-access-control
    type features/access-control
    subvolumes EBSVolume-posix

volume EBSVolume-locks
    type features/locks
    subvolumes EBSVolume-access-control

volume EBSVolume-io-threads
    type performance/io-threads
    option thread-count 4
    subvolumes EBSVolume-locks

volume /mnt/ebs
    type debug/io-stats
    option log-level NONE
    option latency-measurement off
    subvolumes EBSVolume-io-threads

volume EBSVolume-server
    type protocol/server
    option transport-type tcp
    option auth.addr./mnt/ebs.allow *
    subvolumes /mnt/ebs

In our test, all clients starts writing to different 1GB file at the same time.
The measured write bandwidth, with 2x2 servers:

1 client: 6.5 MB/s
2 clients: 4.1 MB/s
3 clients: 2.4 MB/s
4 clients: 4.3 MB/s

This is not acceptable for our needs. With PVFS2 (I know it's
stripping which is very different from replica) we can get up to 35
2-3 times slower than that would be understandable. But 5-15 times
slower is not, and I would like to know whether there is something we
could try out.

Could anybody publish their write speeds on similar setup, and tips
how to achieve better performance?

Thank you,

More information about the Gluster-users mailing list