[Gluster-users] 3.1.2 with "No such file" and "Invalid argument"

[Steve Wilson]

On 01/28/2011 12:49 PM, Steve Wilson wrote:
> I'm running a pair of replicated/distributed GlusterFS 3.1.2 servers, 
> each with 8 bricks.  Here's the command I used to create the data volume:
>
>    gluster volume create post replica 2 transport tcp
>    pablo:/gluster/01/brick stanley:/gluster/01/brick \
>    pablo:/gluster/02/brick stanley:/gluster/02/brick \
>    pablo:/gluster/03/brick stanley:/gluster/03/brick \
>    pablo:/gluster/04/brick stanley:/gluster/04/brick \
>    pablo:/gluster/05/brick stanley:/gluster/05/brick \
>    pablo:/gluster/06/brick stanley:/gluster/06/brick \
>    pablo:/gluster/07/brick stanley:/gluster/07/brick \
>    pablo:/gluster/08/brick stanley:/gluster/08/brick
>
> The clients mount the volume using the gluster FUSE client, again 
> version 3.1.2.  Since GlusterFS doesn't yet support exporting 
> sub-directories directly (it does now for NFS, though, I believe), I 
> mount the volume as:
>     /net/.post-volume
> and then bind mount a couple of lower sub-directories as:
>     /net/post (bound to /net/.post-volume/home)
>     /net/post-scratch (bound to /net/.post-volume/scratch)
>
> I am using this volume for user home directories.
>
> That's the background information... now for the problem.  There are 
> certain files that are almost guaranteed to go missing after less than 
> a few hours of a user starting up a login session.  We use Ubuntu with 
> GDM as a login greeter and both KDE and Gnome desktops.  Here are a 
> few of the files that usually go missing:
>
>    ls: .dmrc: No such file or directory
>    ls: .pine-debug2: Invalid argument
>    ls: .xsession-errors.old: No such file or directory
>    ls: .recently-used.xbel: No such file or directory
>
> Taking .dmrc, for example, I can stop the volume, then remove all 
> occurrences of this file on all the bricks on both servers.  Then 
> start up the volume again, start the user session again, and after a 
> period of time (I'm not sure how long), the file comes up missing 
> again.  When the file is missing, this is what it looks like on the 
> bricks themselves:
>
> server 1:
>
>    root at stanley:/gluster# ls -l ./07/brick/home/stevew/.dmrc
>    ---------T 1 stevew sysmgr 0 2011-01-28 08:24
>    ./07/brick/home/stevew/.dmrc
>    root at stanley:/gluster# ls -l ./02/brick/home/stevew/.dmrc
>    -rw-r--r-- 1 stevew sysmgr 25 2011-01-28 08:25
>    ./02/brick/home/stevew/.dmrc
>
> server 2:
>
>    root at pablo:/gluster# ls -l ./07/brick/home/stevew/.dmrc
>    ---------T 1 stevew sysmgr 0 2011-01-28 08:24
>    ./07/brick/home/stevew/.dmrc
>    root at pablo:/gluster# ls -l ./02/brick/home/stevew/.dmrc
>    -rw-r--r-- 1 stevew sysmgr 25 2011-01-28 08:25
>    ./02/brick/home/stevew/.dmrc
>
>
> Pertinent log messages are:
>
> server 1:
>
>    net-.post-volume.log:[2011-01-28 08:16:28.899544] W
>    [fuse-bridge.c:2432:fuse_xattr_cbk] glusterfs-fuse: 6318974:
>    GETXATTR() /home/stevew/.dmrc => -1 (Invalid argument)
>
>
> server 2:
>
>    net-.post-volume.log:[2011-01-28 08:10:58.74848] W
>    [fuse-bridge.c:2432:fuse_xattr_cbk] glusterfs-fuse: 2428788:
>    GETXATTR() /home/stevew/.dmrc => -1 (No such file or directory)
>
>
> Any thoughts or ideas?
>
> Thanks!
>
> Steve
>
>

Yesterday I disabled the performance.stat-prefetch volume option and 
this has improved the situation greatly.  I can now do a "ls -alR > 
/dev/null" and get no errors on the mounted volume.

Steve