Gluster
  • Documentation
  • Contact
  • About
  • Developers
  • Download

Gluster 3.2: Setting Volume Options

From GlusterDocumentation

You can set volume options, as needed, while the cluster is online and available.

To set volume options

  • Set volume options using the following command:

    # gluster volume set VOLNAME OPTION PARAMETER

    The following lists the options you can set:
cluster.data-self-heal-algorithm      ( full/diff )
cluster.min-free-disk                 ( Required minimum free disk space as a percentage like 30%)
cluster.self-heal-window-size         ( 0 < data-self-heal-window-size < 1025)
cluster.stripe-block-size             (size in bytes )
cluster.quorum-type                   (quorum method)
cluster.quorum-count                  (# needed for quorum)
diagnostics.brick-log-level           ( DEBUG|INFO|WARNING|ERROR|CRITICAL|NONE|TRACE )
diagnostics.client-log-level          (DEBUG|INFO|WARNING|ERROR|CRITICAL|NONE|TRACE )
diagnostics.latency-measurement       ( ON|OFF )
diagnostics.dump-fd-stats             ( ON|OFF )
features.quota-timeout          ( 0 < 3600 secs )
geo-replication.indexing          ( ON|OFF )
network.frame-timeout          ( 1800 secs)
network.ping-timeout          ( 42 secs)
performance.cache-max-file-size       ( size in bytes )
performance.cache-min-file-size       ( size in bytes )
performance.cache-refresh-timeout     ( 0 < cache-timeout < 61 )
performance.cache-size       ( size in bytes )
performance.flush-behind              ( ON|OFF)
performance.io-thread-count   ( 0 < io-threads < 65 )
performance.write-behind-window-size  ( Write-behind cache size )
auth.allow                            ( valid IP address which includes wild card patterns including *, such as 192.168.1.* )
auth.reject
nfs.disable                  (ON|OFF)
nfs.enable-ino32                      (ON|OFF)
nfs.export-dir              (Enable|Disable)
nfs.export-volumes       (ON|OFF)
nfs.addr-namelookup     (ON|OFF)
nfs.register-with-portmap   (ON|OFF)
nfs.port  (38465 to 38467)
nfs.rpc-auth-unix           (ON|OFF)
nfs.rpc-auth-null          (ON|OFF)
nfs.rpc-auth-allow        (IP address or Host name)
nfs.rpc-auth-reject      (IP address or Host name)
nfs.ports-insecure              (ON|OFF)
nfs.trusted-sync        (ON|OFF)
nfs.trusted-write      (ON|OFF)
nfs.volume-access      (read-write|read-only)
rpc-auth-allow-insecure       (ON|OFF)
For example, to specify the performance cache size for test-volume:

# gluster volume set test-volume performance.cache-size 256MB
Set volume successful

The following table lists the Volume options along with its description and default value:

Note: The default options given here are hard coded in the source file and is subject to modification at any given time. The following default values may not be the same for all versions.

Option Description Default Value
cluster.self-heal-window-size
Specifies the number of maximum number blocks per file for which self-heal process would be applied simultaneously. 16
cluster.data-self-heal-algorithm
Selects between "full", "diff", and reset. The "full" algorithm copies the entire file from source to sinks. The "diff" algorithm copies to sinks only those blocks whose checksums don't match with those of source. Reset uses a heuristic model.

If the file does not exist on one of the subvolumes, or a zero-byte file exists (created by entry self-heal) the entire content has to be copied anyway, so there is no benefit from using the "diff" algorithm.

If the file size is about the same as page size, the entire file can be read and written with a few operations, which will be faster than "diff" which has to read checksums and then read and write.

reset
cluster.min-free-disk
Specified the percentage of disk space that must be kept free.

Might be useful for non-uniform bricks.

0%
cluster.stripe-block-size
Specifies the size of the stripe unit that will read from or written to the striped servers.

Optionally different stripe unit sizes can be specified for different files, with the following pattern <filename-pattern:blk-size>.

128 KB (for all files)
cluster.quorum-type Method used for quorum enforcement. "None" means no quorum enforcement, which is the historical behavior. "Auto" means quorum is set to be more than half of the bricks in a subvolume, or exactly half if that includes the first listed brick. "Fixed" means quorum is set to a value specified by cluster.quorum-count. If quorum is not met, then modifing operations such as write will fail with EROFS. This prevents most cases of "split brain" which result from conflicting writes to different bricks. None
cluster.quorum-count Number of subvolumes that must be present to achieve quorum, as described for cluster.quorum-type. This value is not used unless cluster.quorum-type is "fixed". 0
diagnostics.brick-log-level
Changes the log-level of the bricks. INFO
diagnostics.client-log-level
Changes the log-level of the clients. INFO
performance.cache-max-file-size
Sets the maximum file size cached by the io-cache translator. Can use the normal size descriptors of KB,MB,GB,TB or PB (ie. 6GB). Maximum size uint64. 18,446,744,073,709,551,615B
performance.cache-min-file-size
Sets the minimum file size cached by the io-cache translator. Same as "max" above. 0B
performance.cache-refresh-timeout
The cached data for a file will be retained till 'cache-refresh-timeout' seconds, after which data re-validation is performed. 1 sec
network.frame-timeout
The time frame after which the operation has to be declared as dead, if the server does not respond for a particular operation. 1800 (30 mins)
network.ping-timeout
The time duration for which the client waits to check if the server is responsive.

When a ping timeout happens, there is a network disconnect between the client and server. All resources held by server on behalf of the client get cleaned up. When a reconnection happens, all resources will need to be re-acquired before the client can resume its operations on the server. Additionally, the locks will be acquired and the lock tables updated.

This reconnect is a very expensive operation and should be avoided at the cost of client wait time for a network disconnect.

42 sec
auth.allow
'IP addresses/Host name' of the clients which should be allowed to access the the volume. * (allow all)
auth.reject
'IP addresses/Host name' of the clients which should be denied to access the volume. NONE (reject none)
performance.cache-size
Size of the read cache. 32 MB
performance.write-behind-window-size
Size of the per-file write-behind buffer. 1 MB
performance.io-thread-count
The number of threads in IO threads translator. 16
performance.flush-behind
If this option is set ON, instructs write-behind translator to perform flush in background, by returning success (or any errors, if any of previous writes were failed) to application even before flush is sent to backend filesystem. By default, this option is on. ON
diagnostics.latency-measurement
Statistics related to the latency of each operation would be tracked inside GlusterFS data-structures. off
diagnostics.dump-fd-stats
Statistics related to file-operations would be tracked inside GlusterFS data-structures. off
nfs.disable
Turn-off export of volume through NFS off
nfs.enable-ino32
For 32-bit nfs clients or applications that do not support 64-bit inode numbers or large files. Use this

option from the CLI to make Gluster NFS return 32-bit inode numbers instead. Applications that will benefit are those that were either:

  • Built 32-bit and run on 32-bit machines.
  • Built 32-bit on 64-bit systems.
  • Built 64-bit but use a library built 32-bit, especially relevant for python and perl scripts.

This option is disabled by default so NFS returns 64-bit inode numbers by default. Either of the conditions above can lead to application on Linux NFS clients failing with "Invalid argument" or "Value too large for defined data type" errors.

off
nfs.volume-access
Set the access type for the specified sub-volume. read-write
nfs.trusted-write
If there is an UNSTABLE write from the client, STABLE flag will be returned to force the client to not to send a COMMIT request.

In some environments, combined with a replicated GlusterFS setup, this option can improve write performance. This flag allows users to trust Gluster replication logic to sync data to the disks and recover when required. COMMIT requests if received will be handled in a default manner by fsyncing. STABLE writes are still handled in a sync manner.
off
nfs.trusted-sync
All writes and COMMIT requests are treated as async. This implies that no write requests are guaranteed to be on server disks when the write reply is received at the NFS client. Trusted sync includes trusted-write behavior. off
nfs.export-dir
By default, all sub-volumes of NFS are exported as individual exports. Now, this option allows you to export only the specified subdirectory or subdirectories in the volume. This option can also be used in conjunction with nfs3.export-volumes option to restrict exports only to the

subdirectories specified through this option. You must provide an absolute path.

Enabled for all sub directories.
nfs.export-volumes
Enable/Disable exporting entire volumes, instead if used in conjunction with nfs3.export-dir, can allow setting up only subdirectories as exports. on
nfs.rpc-auth-unix
Enable/Disable the AUTH_UNIX authentication type. You must enable this option for better interoperability. However, you can disable it if required. on
nfs.rpc-auth-null
Enable/Disable the AUTH_NULL authentication type. You must always enable this option. on
nfs.rpc-auth-allow<IP- Addresses>
Allow a comma separated list of addresses and/or hostnames to connect to the server. By default, all clients are disallowed. This allows you to define a general rule for all exported volumes. Reject All
nfs.rpc-auth-reject IP- Addresses
Reject a comma separated list of addresses and/or hostnames from connecting to the server. By default, all connections are disallowed. This allows you to define a general rule for all exported volumes. Reject All
nfs.ports-insecure
Allow client connections from unprivileged ports. By default only privileged ports are allowed. This is a global setting in case insecure ports are to be enabled for all exports using a single option. off
rpc-auth-allow-insecure
Allow client connections from unprivileged ports. By default only privileged ports are allowed. This is a global setting in case insecure ports are to be enabled for all exports using a single option. on
nfs.addr-namelookup
Turn-off name lookup for incoming client connections using this option. In some setups, the name server can take too long to reply to DNS queries resulting in timeouts of mount requests. Use this option to turn off name lookups during address authentication. Note, turning this off will prevent you from using hostnames in rpc-auth.addr.* filters. on
nfs.enable-ino32
For NFS clients or apps that do not support 64-bit inode numbers, use this option to make NFS return 32-bit inode numbers instead. Disabled by default so NFS returns 64-bit inode numbers by default. off
nfs.register-with- portmap
For systems that need to run multiple NFS servers, you need to prevent more than one from registering with portmap service. Use this option to turn off portmap registration for Gluster NFS. on
nfs.port <PORT- NUMBER>
Use this option on systems that need Gluster NFS to be associated with a non-default port number. 38465- 38467
geo-replication.indexing
Use this option to automatically sync the changes in the filesystem from Master to Slave. off
features.quota-timeout
For performance reasons, quota caches the directory sizes on client. You can set timeout indicating the maximum duration of directory sizes in cache, from the time they are populated, during which they are considered valid. . 0


 

Copyright © Gluster, Inc. All Rights Reserved.