Building a longer term focus for Gluster

The initial rounds of conversation around the planning of content for release 8 has helped the project identify one key thing – the need to stagger out features and enhancements over multiple releases. Thus, while release 8 is unlikely to be feature heavy as previous releases, it will be the starting point in the set of releases which bring about quite a number of interesting changes to the Gluster project.

While on the topic of releases, it would be ideal to ask users on the release 6 stream to consider moving to newer releases. It allows the project to utilize the available developer capacity to work on fixes in a more updated release. It also enables users to gain from the work being undertaken in release 7 stream (and upcoming release 8) which are more aligned with emerging workloads in hybrid cloud platforms. The project has not taken a drastic decision which changes the current lifecycle for release 6. However, given the available capacity and our focus on bringing improvements in some of the key areas, this would be a decision that is positively opportunistic.

We want to start from release 8 and work through a few releases in order to bring sustainable improvements. These include the possibility of making significant gains for workloads which lookup a huge number of small files or those which invoke volume lifecycle operations at a high rate due to instantiation of project workspaces. These have been some of the harder challenges which have not received a longer term focus from the maintainers. This situation can be changed by choosing to make smaller but more meaningful releases thus providing our community with stability and improvements which create a positive impact on their businesses.

Making changes in the network layer and especially RPC is an involved project. It will require careful planning aligned with a very realistic approach to testing. And as we work through the planning phases of such a change, we will have to put in place decisions around the nature of testing we undertake within the project. We have already kicked off a project which will allow us to shift our development workflow to Github. Having access to more modern infrastructure is likely to reduce the workload on the small team which supports the infrastructure for the Gluster project. The adoption of workflows which are similar to other projects would help us enable a governance structure that makes for a more predictable software supply pipeline. To achieve this, we will have to bolster our plans for testing in a distributed architecture.

The topic of improving testing is not limited to adding new tests and wider test coverage. It also includes the long term future of an automated test framework which lends itself well to new workloads as well as integrating fully with Github driven software development workflows. The Glusto project has in the past demonstrated promise and potential. However, the project has been unable to show the full extent of value within the Gluster project. Absence of significant project activity and long periods of no-shows by the maintainers would give the impression that it would be in Gluster’s interest to search for a more integrated framework to enable distributed testing. As complementary projects such as Kadalu.io aim to work around the constraints of glusterd based management framework, the testing practices would need to adopt some of the ideas being discussed and evaluate them for implementation.

Improvements in the logging and monitoring components are not based on creating our own stovepipe or, stack. The Gluster project would be focused on more integration and service endpoints which allow better consumption by existing tools which are designed for that particular function. As an example, working on better integration with Prometheus based monitoring flows to expose a better set of metrics for analysis. There are plenty of opportunities to bring about changes and improvements which aid instrumentation and align with the best practices of an established field such as observability (o11y). By making Gluster more durable for rapid deployment cycles demanded in SRE/Ops designs, we will be able to improve the time-to-recovery.

We will continue to improve Gluster with focus on the long term. Our choice would be influenced by an increasing set of input from our users. By moving away from big targets into much more manageable release sets we hope to improve the quality of software and thus continue to enable new users and workloads in the hybrid cloud focused IT infrastructure. Conversations – with users, with members from other communities and mostly amongst ourselves is the foundation on which we can commit to contributions which ensure a more resilient future for the project. To that end, we will have to review and revisit the choices we have made in the past and be bold to decide if they need modifications to address the challenges we see ahead of us.