DownloadBooster Configuration Reference Page
From GlusterDocumentation
Contents |
Introduction
This page describes how to write a configuration file for booster. Before we we dive into the details, it must be clarified that as of release 2.0.2, only a specific subset of POSIX FS related system calls are implemented in booster. However, the ones that have been implemented would, we feel, cover almost all the requirements of an application using the file system.
The list of system calls that are present are:
| open | open64 | creat | close | read | readv | pread | pread64 |
| write | writev | pwrite | pwrite64 | lseek | lseek64 | dup | dup2 |
| fork | mkdir | rmdir | chmod | chown | fchmod | fchown | rename |
| fsync | ftruncate | link | utimes | utime | mknod | mkfifo | unlink |
| symlink | readlink | realpath | opendir | readdir | closedir | readdir64 | stat |
| fstat | lstat | statfs | statvfs | telldir | rewinddir | seekdir | lchown |
| sendfile | readdir_r | fcntl (not all fcntls) |
Remaining APIs are continuing to be implemented.
Virtual Mount Points
All applications need to start working from a particular path in the file system hierarchy. Moreover, to perform any operation on a file they must have the path to the file, either relative or absolute. Unlike traditional file system system APIs, which must start operating on a path whose first component is actually a directory on the local disk. In simpler words, the mount point for a file system should be a local directory. An application then operates on the files and directories within this mount point even though these files and directories could exist on a remote system. For example, in the case of an NFS mount.
Since booster transparently redirects an application's file system operations, it is important that the redirection of file system operations takes into account the parts of the file system over which the application is going to run. Because booster does not currently support storing per-process current working directory information, an absolute path needs to be specified as a Virtual Mount Point or VMP over which the application will be performing its operations. A VMP is not a real directory on the disk, rather it is merely an identifier in guise of an absolute path to a directory. The idea is to clearly demarcate the part of the file system name space over which if the application issues an FS requests, that request will be directed into GlusterFS. Of course, there will have to be some way of ensuring that the user of the application understands the existence of this boundary.
Note that LD_PRELOADing booster does not stop local file system requests from being forwarded to the actual files and directories on disk. Only those file system operations that were initiated using a path pre-fixed with the VMP are redirected into GlusterFS.
booster.conf
The association between a VMP and the corresponding GlusterFS Volume Specification File is established through a configuration file that is similar in format to the /etc/fstab on Unixen.
For example, a traditional fstab file might look like:
| /dev/sda1 / ext3 defaults,errors=remount-ro 0 1 |
Where /dev/sda1 is the block device over which an ext3 file system is mounted at /. The comma-separate components following ext3 are the options handed to the file system code.
Similar formatted lines are used in a booster configuration file, as shown below:
| /docs/volfiles/localdiskiotclient.vol /testpath/ glusterfs subvolume=localdiskthreads,logfile=/test/clog,loglevel=DEBUG |
Where the block device name has been replaced by the absolute path to the volume specification file. The virtual mount point, like the mount point / for the earlier example, is /testpath/. Followed by comma-separated options that are required by the booster library. These options are:
| subvolume | Specifies the name of the particular subvolume to use as root from the volume specification file. |
| logfile | Specifies the absolute path to the log file where booster library will write out messages relevant to its operation. |
| loglevel | The log level to use when writing messages to log files. These log levels are the same as the standard log levels defined for GlusterFS. |
| attr_timeout | The duration in seconds, for which a file or directory's attributes like file size, blocks, etc, are valid in the attribute cache.
|
Effectively, the specification above tells booster, that any file system operation that it receives for a path starting with /testpath/ as the root directory should be performed over the GlusterFS volume called localdiskthreads. Operations using all other paths are forwarded to libc where they are handled like a regular system call.
Mount Point Bypass
Mount point bypass is the new name for the old approach of performing operations through booster. The idea was that an application accessing a GlusterFS server could do so without even a configuration file as required by the VMP-based approach. This simplicity, however, came at a cost.
- That of the need to perform a few extra system calls in order to extract some state from the GlusterFS mount point for initializing the corresponding booster state.
- Support for only file descriptor based operations and not path-based ones(..except open() and creat() since they have to return the file descriptors..).
The second drawback can be overcome but at the cost of increased complexity in booster. So we decided against implementing this functionality for now.
We feel that the VMP-based approach is simpler and at the same time avoids all file system related system calls since all FS operations happen through libglusterfsclient, a user-space library extension for the GlusterFS client side stack.
Still, applications that need to use the old approach, can do so by simply mounting GlusterFS and pointing the application to perform its file system operations over that mount point. In order to utilize the mount-point bypass approach, booster will have to be LD_PRELOAD'ed using the instructions below but without requiring the booster configuration file or setting the environment variables.
Application Execution
Now that we've setup the association above between a volume spec and a VMP, it is now time to run the application. Before running the application, it is necessary to define two environment variables for the application.
These are:
| GLUSTERFS_BOOSTER_FSTAB | Must be set to the absolute path to the configuration file created above. |
| LD_PRELOAD | Must be set to the absolute path to the booster library file. |
Suppose the booster configuration file is at /docs/booster.conf and the booster library has been installed in /usr/local/lib/glusterfs/glusterfs_booster.so. Now the commands to run the application would be:
| $ export GLUSTERFS_BOOSTER_FSTAB="/docs/booster.conf" $ LD_PRELOAD="/usr/local/lib/glusterfs/glusterfs_booster.so" <APP-BINARY> |
