DownloadUnfs3boosterConfiguration
From GlusterDocumentation
Contents |
Introduction
FIXME: change file title to "Adding unfs3" The contents of this file will be modified for CIFS and WebDAV
GlusterFS uses unfs3 as a way to provide NFS export for GlusterFS volumes without the over-head of going through the FUSE layer. This is done by allowing unfs3 to operate over the booster library, which is a direct access library used to re-route file system operations into a GlusterFS volume. Booster allows the GlusterFS client-side stack and unfs3 to reside in the same address space, which avoids the need for kernel entry for every file system operation.
FIXME: Fix link when filename changes.
Before adding unfs3, follow the steps in Installing GlusterFS on the Server.
Configuration
The first step is to install a slightly patched copy of unfs3. It contains a few bug-fixes as well as small changes that are needed for working over booster.
- Get the latest unfs3booster release:
- Install unfs3booster.
- Install using an RPM by entering the following command:
- Install using source tarball by entering the following commands:
- Create an exports file to use with the unfs3 server. The syntax for the unfs3 file is similar to the Linux NFS server /etc/exports file, but with fewer options. (See unfsd(8) for a list of all supported options.) In the exports file, each line specifies an exported directory along with the options that control this export. The following are some common sample unfs3 config lines:
- Create a booster configuration file.
- Create an exports file in /etc/unfs3exports on the NFS server.
- Create simple server volume file.
- Create a simple client volume file.
- Create a /etc/booster.fstab file on the NFS server.
- Define the required environment variables on the NFS server.
- Start GlusterFS server on brick1:
- Start unfs3 on the NFS server machine.
- Mount the GlusterFS volume through NFS on a client machine.
| System | URL |
| x86_64, CentOS, RHEL 5 | http://ftp.gluster.com/pub/gluster/glusterfs/misc/unfs3/0.5/unfs3-0.9.23booster0.5-1.el5.x86_64.rpm |
| x86_64, SLES 11 | Coming Soon |
| Source Tarball | http://ftp.gluster.com/pub/gluster/glusterfs/misc/unfs3/0.5/unfs3-0.9.23booster0.5.tar.gz |
| Source RPM | http://ftp.gluster.com/pub/gluster/glusterfs/misc/unfs3/0.5/unfs3-0.9.23booster0.5-1.el5.src.rpm |
$ rpm -ivh unfs3-0.9.23booster0.5-1.el5.x86_64.rpm (CentOS/RHEL 5)
$ tar -xzf unfs3-0.9.23booster0.5.tar.gz
$ cd unfs3-0.9.23booster0.5/
$ ./bootstrap
$ ./configure
$ make;make install
#Only one client and allow both read-write. /exports 192.168.1.201(rw)
#Do not re-map root UID to anonymous user. /data/maildirs 192.168.1.202(rw,no_root_squash)
#Allow clients from "trusted" to mount read-write without root UID squashing. #For all others, allow only read-only. /home trusted(rw,no_root_squash) (ro)
#Allow machine "weirdo" to mount export directory using default options. "/with spaces" weirdo
#Allow one specific client read-write access and allow machines from #a subnet read-only access while squashing all UIDs to anonymous user. /usr 1.2.3.4(rw) 192.168.2.0/24(ro,all_squash)
#Another way of specifying netmask. /home/foo bar(rw) 10.0.0.0/255.0.0.0(root_squash)
#Allow explicit specification of the anonymous UID. /home/joe joes_pc(anonuid=1100,anongid=1100,rw,all_squash)
This configuration file establishes an association between a mount point and the GlusterFS volume specification for accessing GlusterFS storage bricks through this mount-point.
The following instructions show how to write a fstab file for booster. (The booster configuration file has the same format as the Linux /etc/fstab file.)
Like the NFS exports file, the booster.fstab file describes one volume per line.
For example, the following line tells booster to perform file system operations that include /testpath/ as the root directory over the GlusterFS volume called localdiskthreads:
/docs/volfiles/localdiskiotclient.vol /testpath/ glusterfs subvolume=localdiskthreads,logfile=/test/clog,loglevel=DEBUG,attr_timeout=0
For details on the comma-separated options in the last field of the line, see BoosterConfiguration.
Starting the unfs3 Server
FIXME: My description isn't very good. Can you help? Was: instructions towards running GlusterFS and unfs3booster.
This section describes how to configure and start GlusterFS and unfs3 server.
FIXME: should the following be unfs3--not NFS?? "NFS server" is fine.
/testpath 192.168.0.0/16(rw,no_root_squash)
For example, see the following /test/docs/server.vol file on the GlusterFS server:
volume localdisk
type storage/posix
option directory /data/export
end-volume
volume localdisk-locked
type features/locks
subvolumes localdisk
end-volume
volume localdisk-locked-iot
type performance/io-threads
subvolumes localdisk-locked
option thread-count 8
end-volume
volume localdisk-server
type protocol/server
option transport-type tcp
option transport.socket.listen-port 2703
subvolumes localdisk-locked-iot
option auth.addr.localdisk-locked-iot.allow *
end-volume
For example, see the following on /test/docs/client.vol on the NFS server.
volume diskclient
type protocol/client
option transport-type tcp
option remote-port 2703
option remote-host brick1
option remote-subvolume locadisk-locked-iot
end-volume
volume disk-wb
type performance/write-behind
subvolumes diskclient
end-volume
This tab ties the client volfile above to a mount point that booster uses. For example:
/test/docs/client.vol /testpath/ glusterfs subvolume=localdiskthreads,logfile=/test/clog,loglevel=DEBUG,attr_timeout=0
Note: you configure this in the client.vol file because unfs3booster is a client of GlusterFS. The first field in italics in the above line must be same as the path exported through the exports file. The second italics field above must be the name of subvolume that is exported through unfs3. This subvolume must exist in the client volume spec.
The following using an environment variable tells booster about the location of its booster.fstab file.
$ export GLUSTERFS_BOOSTER_FSTAB=/etc/booster.fstab
$ glusterfsd -f /test/docs/server.vol -L NONE -l /test/srvlog
(If you changed the path to the glusterfs-booster.so library was changed at installation, use the current path.)
$ LD_PRELOAD=/usr/local/lib/glusterfs/glusterfs-booster.so unfsd -e /etc/unfs3exports
$ mount nfsserver:/testpath/ -o wsize=65536,rsize=65536 /mnt
Further Information
Due to a restriction in the GlusterFS 2.0.2 release, the NFS wsize and rsize parameters cannot exceed 128 KB. (This will be fixed soon.) Therefore, even though unfs3 allows 512 KB as the maximum value for these parameters, ensure that the client mount the unfs3booster NFS exports using a value less than or equal to 128 KB. For example, on Linux:</li>
$ mount 192.168.1.10:/boosterexport/ -o wsize=65536,rsize=65536 /mnt
Note: The changes to unfs3 are generic enough to use for non-GlusterFS file systems also. There is no need to setup two separate versions of unfs3 or separate export files.
