[Gluster-users] IPoIB Volume (3.3b3) started but not online, not mountable
landman at scalableinformatics.com
Wed Apr 18 16:04:26 PDT 2012
On 04/18/2012 06:58 PM, Harry Mangalam wrote:
> And one more observation that will probably be obvious in retrospect. If
> you enable auth.allow (on 3.3b3), it will do reverse lookups to verify
> hostnames so it will be more complicated to share an IPoIB gluster
> volume to IPoEth clients.
> I had been overriding DNS entries with /etc/hosts entries, but the
> auth.allow option will prevent that hack.
> If anyone knows how to share an IPoIB volume to ethernet clients in a
> more formally correct way, I'd be happy to learn of it.
After dealing with problems in multi-modal networks with slightly
different naming schemes, I don't recommend using tcp and RDMA together
(or even IPoIB with eth) for Gluster. Very long, very painful saga.
Executive summary: here be dragons.
Also, IPoIB is very leaky. So under heavy load, you can find your
servers starting to run out of memory. We've seen this with OFED
through 1.5.3.x and Glusters as late as 3.2.6.
We'd recommend sticking to one fabric for the moment with Gluster. Use
real tcp with a 10 or 40 GbE backbone. Far fewer problems. Much less
Joseph Landman, Ph.D
Founder and CEO
Scalable Informatics Inc.
email: landman at scalableinformatics.com
web : http://scalableinformatics.com
phone: +1 734 786 8423 x121
fax : +1 866 888 3112
cell : +1 734 612 4615
More information about the Gluster-users