Some notes on libgfapi.so symbol versions in GlusterFS 3.6.1

A little bit of background——

We started to track API/ABI changes to libgfapi.so by incrementing the SO_NAME, e.g. libgfapi.so.0(.0.0). In the master branch it was incremented to to ‘7’ or libgfapi.so.7(.0.0) for the eventual glusterfs-3.7.

I believe, but I’m not entirely certain¹, that we were supposed to reset this when we branched for release-3.6. Reset it to either ‘6’ or ‘0’, but we didn’t — apparently we forgot about it. In the 3.6.0 betas and if you build the GA release of 3.6.0 you’ll get a libgfapi.so.7(.0.0).

We didn’t hear much, if anything about this until a few days before 3.6.0 was scheduled to GA, when we were ‘reminded’ that older versions of applications like qemu, Samba, and more — linked against previous versions of libgfapi.so — no longer worked after upgrading to the new version of glusterfs.

We briefly experimented with adding a -compat package that installed a symlink: libgfapi.so.0 -> libgfapi.so.7; but some thought this was too much of a hack, and we abandoned that idea.

As a result we now have symbol versions in libgfapi.so. I’ve posted a public spreadsheet with a table of the symbols and the versions of glusterfs that they appear in at

https://docs.google.com/spreadsheets/d/1SKtzgEeVZbKFJgGGMdftf0p-AB5U7yyhVq1n2b6hBeQ/edit?usp=sharing

and also at

https://ethercalc.org/lrjvqrapzu

There are a few things to note about the symbol versions:

1. so far all the function signatures, i.e. number of parameters and their types have not changed since libgfapi was introduced in 3.4.0. That’s a Good Thing.
2. the symbol versions are taken from the (community) glusterfs release that they first appeared in.
3. there are two functions declared in glfs.h that do not have an associated definition. So far it’s not clear why.
4. there are two functions defined (in glfs-fops.c) that look like they ought to have declarations in glfs.h. Perhaps this was an oversight in the original implementation.
5. there are several (private?) functions in libgfapi that are not declared in a public header that are used/referenced outside the library. That’s not a Bad Thing, per se, but it’s also not a Good Thing. It seems a bit strange for, e.g. glfs-heal and the nfs server xlator to have to be linked against libgfapi.so. These functions should either be made public or moved to another library, e.g. libglusterfs.so

N.B. that 3, 4, and 5 are also noted in comments in the spreadsheets.

In parallel to all this, RHEL 6.6, RHEL 7.0, and the related CentOS releases shipped updated RHS-GlusterFS client-side packages with version 3.6.0.X. This has resulted in confusion for many of the users of Community GlusterFS who are having issues with upgrading their systems. On top of that the libgfapi.so in these releases is libgfapi.so.0(.0.0), and there are applications included in those releases that are linked with it.

Earlier today (7 Nov, 2014) we released GlusterFS 3.6.1 to address and hopefully mitigate the 3.6.0 upgrade issue and fix the libgfapi.so compatibility issue by reverting to the original SO_NAME (libgfapi.so.0.0.0), now with symbol versions. The applications will not need to be recompiled or relinked.

Knowing that we were going to quickly release GlusterFS 3.6.1 to address these issues, to save our community packagers some work and to try to minimize confusion² we agreed in the community to not package GlusterFS 3.6.0 for any of the Linux distributions. We expect packages for 3.6.1 to be available soon on download.gluster.org.

And if anyone is looking for a nice Google Summer of Code project, linking libglusterfs and the xlators with link maps — with or without symbol versions — is an idea that I think has some merit.

HTH.

¹Not without slogging through a lot of old emails to reconstruct what we originally intended.

²Then we don’t have to explain why some Linux distributions have community 3.6.0 packages and others have (only) 3.6.1.