[Gluster-users] strange problem

[Vikas Gorur]

On Jan 12, 2011, at 1:25 AM, Koleszár Ádám wrote:

> Hi,
> 
> I have a strange problem. I am using glusterfs 3.0.5 on 3 machines with CentOS 5.4. If I am using the mounted glusterfs intensively it is using more and more cache. I think it is normal. But as more and more cache used, it's getting slower and slower.
> The glusterfsd process uses 500-700% CPU (normal 1-10%, btw it's a 8 core machine). On the mounted glusterfs a simple directory change takes about half minute, and every operation very very slow on the mounted glusterfs.
> 
> If i execute the following command:
> sync && echo 3 > /proc/sys/vm/drop_caches
> 
> Almost the entire system memory freed, and glusterfsd CPU usage fall back 1-10%, and everything working fine.. for a while
> 
> Now, i have to put this command into cron for every night.
> 
> Has someone encountered the same problem?
> Is there any other solution?


You can set the sysctl parameter vfs_cache_pressure to a high value to make the kernel free up the cache at a higher rate. We've seen better latencies with this set to something like 10000. You can also experiment with setting vm.swappiness to 0.

/etc/sysctl.conf

vm.vfs_cache_pressure=10000
vm.swappiness=0

# sysctl -p

------------------------------
Vikas Gorur
Engineer - Gluster, Inc.
------------------------------