[Gluster-users] Mac / NFS problems

Tue Mar 15 10:46:14 UTC 2011

well, answering our own question;  it seems that NFS on the mac (10.6.6) has
become problematic due to the increased amount of NFS locking used.  you
just mount with nolocks and things start working.  i hope this help someone
else out there...

regards,

paul

quoting from http://www.facebook.com/note.php?note_id=125738946623

I don't know what it is about Apple and NFS, but they keep moving things
around. The new UI to NFS mounting is much nicer than it was before, but
it's now in a totally different place: the Disk Utility. But if you use a
lot of NFS file systems, it's a pain to have to mount them one by one:
ignoring the UI and using the /net automount filesystem is far more
convenient. Just use the file name /net/hostname/path and you don't have to
mess with any mounting, it just happens by automagic. I wrote a blog entry
about this a long time ago.
However, there is a huge problem with this: OS X does a phenominal amount of
file locking (some would say, needlessly so) and has always been really
sensitive to the configuration of locking on the NFS servers. So much so
that if you randomly pick an NFS server in a large enterprise, true success
is pretty unlikely. It'll succeed, but you'll keep getting messages
indicating that the lock server is down, followed quickly by another message
that the lock server is back up again. Even if you do get the NFS server
tuned precisely the way that OS X wants it, performance sucks because of all
the lock/unlock protocol requests that fly across the network. They clearly
did something in Snow Leopard to aggravate this problem: it's now nasty
enough to make NFS almost useless for me.

Fortunately, there is a fix: just turn off network locking. You can do it by
adding the "nolocks,locallocks" options in the advanced options field of the
Disk Utility NFS mounting UI, but this is painful if you do a lot of them,
and doesn't help at all with /net. You can edit /etc/auto_master to add
these options to the /net entry, but it doesn't affect other mounts -
however I do recommend deleting the hidefromfinder option in auto_master. If
you want to fix every automount, edit /etc/autofs.conf and search for the
line that starts with AUTOMOUNTD_MNTOPTS=. These options get applied on
every mount. Add nolocks,locallocks and your world will be faster and
happier after you reboot.

On 11 March 2011 09:52, Shehjar Tikoo <shehjart at gluster.com> wrote:

> David Lloyd wrote:
>
>> Hello,
>>
>> Were having issues with macs writing to our gluster system.
>> Gluster vol info at end.
>>
>> On a mac, if I make a file in the shell I get the following message:
>>
>> smoke:hunter david$ echo hello > test
>> -bash: test: Operation not permitted
>>
>>
> I can help if you can send the nfs.log file from the /etc/glusterd
> directory on the nfs server. Before your mount command, set the log-level to
> trace for nfs server and then run the echo command above. Unmount as soon as
> you see the error above and email me the nfs.log.
>
> -Shehjar
>
>
>
>
>> And the file is made but is zero size.
>>
>> smoke:hunter david$ ls -l test
>> -rw-r--r--  1 david  realise  0 Mar  3 08:44 test
>>
>>
>> glusterfs/nfslog logs thus:
>>
>> [2011-03-03 08:44:10.379188] I [io-stats.c:333:io_stats_dump_fd]
>> glustervol1: --- fd stats ---
>>
>> [2011-03-03 08:44:10.379222] I [io-stats.c:338:io_stats_dump_fd]
>> glustervol1:       Filename : /production/hunter/test
>>
>> Then try to open the file:
>>
>> smoke:hunter david$ cat test
>>
>> and get the following messages in the log:
>>
>> [2011-03-03 08:51:13.957319] I [afr-common.c:716:afr_lookup_done]
>> glustervol1-replicate-0: background  meta-data self-heal triggered. path:
>> /production/hunter/test
>> [2011-03-03 08:51:13.959466] I
>> [afr-self-heal-common.c:1526:afr_self_heal_completion_cbk]
>> glustervol1-replicate-0: background  meta-data self-heal completed on
>> /production/hunter/test
>>
>> If I do the same test on a linux machine (nfs) it's fine.
>>
>> We get the same issue on all the macs. They are 10.6.6.
>>
>> Gluster volume is mounted:
>> /n/auto/gv1             -rw,hard,tcp,rsize=32768,wsize=32768,intr
>> gus:/glustervol1
>> Other nfs mounts on mac (from linux servers) are OK
>>
>> We're using LDAP to authenticate on the macs, the gluster servers aren't
>> bound into the LDAP domain.
>>
>> Any ideas?
>>
>> Thanks
>> David
>>
>>
>> g3:/var/log/glusterfs # gluster volume info
>> Volume Name: glustervol1
>> Type: Distributed-Replicate
>> Status: Started
>> Number of Bricks: 4 x 2 = 8
>> Transport-type: tcp
>> Bricks:
>> Brick1: g1:/mnt/glus1
>> Brick2: g2:/mnt/glus1
>> Brick3: g3:/mnt/glus1
>> Brick4: g4:/mnt/glus1
>> Brick5: g1:/mnt/glus2
>> Brick6: g2:/mnt/glus2
>> Brick7: g3:/mnt/glus2
>> Brick8: g4:/mnt/glus2
>> Options Reconfigured:
>> performance.stat-prefetch: 1
>> performance.cache-size: 1gb
>> performance.write-behind-window-size: 1mb
>> network.ping-timeout: 20
>> diagnostics.latency-measurement: off
>> diagnostics.dump-fd-stats: on
>>
>>
>>
>>
>>
>>
>>
>> ------------------------------------------------------------------------
>>
>>
>> _______________________________________________
>> Gluster-users mailing list
>> Gluster-users at gluster.org
>> http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
>>
>
> _______________________________________________
> Gluster-users mailing list
> Gluster-users at gluster.org
> http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
>