GlusterFS General FAQ
What is GlusterFS?
GlusterFS is a clustered file-system capable of scaling to several peta-bytes. It aggregates various storage bricks over Infiniband RDMA or TCP/IP interconnect into one large parallel network file system. Storage bricks can be made of any commodity hardware such as x86-64 server with SATA-II RAID and Infiniband or GigE interconnect.
What is the problem with existing Cluster File Systems?
Most existing cluster file systems are not mature enough for the enterprise market. They are too complex to deploy and maintain, although they are extremely scalable and cheap since they can be entirely built out of commodity OS and hardware.
GlusterFS solves this problem. GlusterFS is an easy to use clustered file system that meets enterprise-level requirements.
Why is striping bad?
- Increased Overhead
Striping files across multiple bricks and reading/writing them at same time will cause serious disk contention issues and the performance will suffer badly as load increases. If you avoid striping, the underlying filesystem and the I/O scheduler on each brick knows best how to organize the file data into contiguous disk blocks for optimized read and write operations.
- Increased Complexity
Striping complicates the design of clustered filesystems badly. Instead of using the underlying mature filesystem's ability to do block disk management, you will have to implement another clustered file system across multiple underlying filesystems (duplication).
- Increased Risk
Loss of a single node can mean loss of entire file system. Imagine how slow it is to run fsck on hundreds of TBs of data.
In reality, striping introduces more problems than it solves, particularly when a file system scales beyond hundreds of TBs.
Alternatively when files and folders remain as is, they take advantage of the underlying file system to do the real block I/O management. A single file can grow from 4TB to 16TB within a single node. In reality, files are not of TBs in size. When multiple clients access a same file, most likely the blocks are already cached in the RAM and sent via RDMA to the clients. GlusterFS takes advantage of high bandwidth low-latency interconnects such as Infiniband.
The GlusterFS AFR (Automatic File Replication) translator does a striped read to improve performance on mirrored files.
You say striping is bad, but I see a 'cluster/stripe' translator in GlusterFS, why is it?
Well, let's list a few points.
- Implementing the striping feature was a few days work for the GlusterFS team, due to its modular design.
- There are a few people/companies who want a striping feature in the FS as their application uses a single large file (100GB - 2TB).
- If the file is very big (which we don't recommend, though) a single server gets completely loaded when its accessed, so striping helps to distribute the load.
- If one uses 'cluster/afr' translator with 'cluster/stripe' then GlusterFS can provide high availability.
Advantages of GlusterFS
- Designed for O(1) scalability and feature rich.
- Aggregates on top of existing filesystems. User can recover the files and folders even without GlusterFS.
- GlusterFS has no single point of failure. Completely distributed. No centralized meta-data server like Lustre.
- Extensible scheduling interface with modules loaded based on user's storage I/O access pattern.
- Modular and extensible through powerful translator mechanism.
- Supports Infiniband RDMA and TCP/IP.
- Entirely implemented in user-space. Easy to port, debug and maintain.
- Scales on demand.
Can GlusterFS store files redundantly?
GlusterFS automatic-file-replication translator does the job.
Minimum system requirements to use GlusterFS?
On the client side, GlusterFS requires FUSE (Filesystem in Userspace http://fuse.sf.net) kernel support. There are no minimum system requirements as such. It can even run within a system on local loopback. However, here are a few suggestions.
- Storage Cluster:
A cluster of 4 or more x86-64 servers with SATA-II RAID and Infiniband interconnect.
- NAS Server:
x86-64 platform, 2GB RAM, SATA-II RAID and gigabit ethernet network interface card.
You can vary the amount of RAM, type of interconnect (say Infiniband or GigE), number of processors, number of bricks, amount of storage capacity per brick, type of disk (say Ultra320 SCSI or SATA II)... all depending upon your application needs.
What operating systems are supported by GlusterFS?
Please refer to "Operating System Requirements"- 
Will a client for Windows be made?
Currently the support is through CIFS, (samba export). But we do have it in mind to port it natively using WinFUSE. The work will be undertaken once WinFUSE is stable
What file system semantics does GlusterFS Support; is it fully POSIX compliant?
GlusterFS is fully POSIX compliant.
Is GlusterFS like parallel NFS?
From the user's point of view, YES, but a very different design internally. The NFS protocol is very old and not very efficient. It is hard to fix or improve it. The Parallel NFS client is anyway incompatible with the existing NFS protocol itself. That's why we moved ahead with a new GlusterFS file system implementation. GlusterFS has an edge over NFS in both features and performance depending on the workload.
Is the gluster client or NFS client faster?
It depends on the use case. For reading many small files, i.e. PHP web serving, the NFS client will perform much better. If the work-load is write-heavy, the native client will perform better.
Is GlusterFS compatible with NFS or SAMBA?
No. But you can re-export NFS / Samba / CIFS on top of GlusterFS though.
Will GlusterFS be upward compatible?
Q: Since GlusterFS is so easy to scale, and after large deployments are setup, when new versions of Gluster come out, will I have to bring down my GlusterFS to upgrade, or will Gluster work in a heterogeneous environment where part of the servers can be upgraded to the newer version and some of the servers run older versions?
A: It's true that with newer versions, we are going to introduce new features. But, considering situations like this, we will be handling such situations (backward compatibility) but the user may miss the new features altogether. Hence we recommend that one upgrades to newer versions. Anyway, we will announce the compatibility break in mailing list.
The GlusterFS protocol has a version checking mechanism, which checks for the compatibility of both server and client before making a successful connection.
NOTE: Its very much advised to use the same version of client and server.
How to interpret GlusterFS versions?
- MAJOR changes when compatibility is broken. You can imagine NFSv2, NFSv3 or Ext2, Ext3 and Ext4 filesystems.
- MINOR changes for every significant release and then goes into a maintenance phase.
- PATCH-LEVEL changes within a maintenance phase for bug fixes or insignificant changes.
Can I add my question here?
If you do not find your question answered here and if you think it is a frequently asked question, please drop us an email dl-docs(at)gluster.com.