GlusterFS communicates over TCP. This allows for stateful handling of file descriptors and locks. If, however, a server fails completely, kernel panic, power loss, some idiot with a reset button… the client will wait for ping-timeout (42 by the defau…

I had a computer with some bad ram that created a corrupt index in Elasticsearch. After trying all weekend and half of Monday to figure out how to get Elasticsearch running again with no response on the IRC channel, I eventually figured it out with the…

I have generic users that are logged in automatically since they’re not allowed user prefs anyway, and it would be more of a hindrance in their environment.
I use puppet to manage a staging directory that has the home directory as I want it to be, and …

Somebody today asked if GlusterFS could be made as fast as a local filesystem. My answer just came out without a ton of thought behind it, but I found it rather profound.
Comparing (any) clustered filesystem to a local filesystem is like comparing appl…

In mixed results, some users have been reporting issues with mounting GlusterFS volumes at boot time. I spun up a VM at Rackspace to see what I could see.
For my volume I used the following fstab entry. The host is defined in /etc/hosts:
server1:testvo…

Here’s a nice post about creating a linked list topology for a distributed-replicated setup. The idea is that it is easier to add a single server to a replicated volume by spending a bit of extra time prepping a linked list of bricks. The default topology would leave the author with the need of adding …Read more

Last month, I received two new servers to replace two of our three (replica 3) GlusterFS servers. My first inclination was to just down the server, move the hard drives into the new server, re-install the OS (moving from 32 bit to 64 bit), and voila, d…

This has come up several times in the last week. “I have 2n servers with 2 or 4 bricks each and I want to add 1 more server. How do I ensure the new server isn’t a replica of itself?”
This isn’t a simple thing to do. When you add bricks, replicas are a…

This sets up a GlusterFS Unified File and Object (UFO) server on a single node (single brick) Gluster server using the RPMs contained in my YUM repo at http://repos.fedorapeople.org/repos/kkeithle/glusterfs/. This repo contains RPMs for Fedora 16, Fedora 17, and RHEL 6. Alternatively you may use the glusterfs-3.4.0beta1 RPMs from the GlusterFS YUM repo at http://download.gluster.org/pub/gluster/glusterfs/qa-releases/3.4.0beta1/ …Read more

GlusterFS spreads load using a distribute hash translation (DHT) of filenames to it’s subvolumes. Those subvolumes are usually replicated to provide fault tolerance as well as some load handling. The advanced file replication translator (AFR) departs f…

On Sunday, March 18th, Fan Yong commited a patch against ext4 to “return 32/64-bit dir name hash according to usage type”. Prior to that, ext2/3/4 would return a 32-bit hash value from telldir()/seekdir() as NFSv2 wasn’t designed to accomidate anything…

A GlusterFS user from IRC asked me about my puppet management of KVM in RHEL/CentOS and how it works. I started to write this post two weeks ago and had to stop because although it works great, I figured that wasn’t the answer he was looking for. I loo…

(This was originally posted on our Q&A site at community.gluster.org) Problem: VERY slow performance when using ‘bedtools’ and other apps that write zillions of small output chunks. If this was a self-writ app or an infrequently used one, I wouldn’t bother writing this up, but ‘bedtools’ is a fairly popular genomics app and since many …Read more

With the addition of automated self-heal in GlusterFS 3.3, a new hidden directory structure was added to each brick: “.glusterfs”. This complicates split-brain resolution as you now not only have to remove the “bad” file from the brick, but it’s counte…

Starting with GlusterFS 3.3, one change has been the check to see if a directory (or any of it’s ancestors) is already part of a volume. This is causing many support questions in #gluster.
This was implemented because if you remove a brick from a volum…

More often than I would like, someone with twenty or more web servers servicing tens of thousands of page hits per hour comes into #gluster asking how to get the highest performance out of their storage system. They’ve only just now come to the realiza…

Since GlusterFS is fuse based, it can be mounted as a standard user without too much difficulty.
On a server:
gluster volume set $VOLUME allow-insecure on
On the client as root:
echo user_allow_other >> /etc/fuse.conf
To mount the volume, you…