• Personal tools
  • Log in / create account
• Navigation
  • Gluster 3.2 Documentation
  • Gluster 3.1.x Documentation
  • Gluster 3.0.x Documentation
  • GlusterFS Concepts
  • Older Versions
  • Recent changes
• Toolbox
  • What links here
  • Related changes
  • Upload file
  • Special pages

Development Work Flow

From GlusterDocumentation

Contents 1 Development work flow of Gluster 1.1 Basics 1.1.1 Git 1.1.2 Gerrit 1.1.3 Jenkins 1.2 Preparatory Setup 1.2.1 Register 1.2.2 Preferred email 1.2.3 Set Username 1.2.4 Watch glusterfs 1.2.5 Email filters 1.2.6 SSH keys 1.2.7 Clone a working tree 1.3 Branching policy 1.3.1 Master/release branches 1.3.2 Personal per-task branches 1.4 Commit policy 1.5 Review process 1.5.1 rfc.sh 1.6 Auto verification 1.7 Reviewing / Commenting 1.8 Incorporate, Amend, rfc.sh, Reverify 1.9 Submission Qualifiers

Development work flow of Gluster

This document provides an overview of the development model followed by the GlusterFS project. It largely revolves around the features and functionality provided by Git version control system, Gerrit code review system and Jenkins continuous integration system. It is a primer for a contributor to the project.

Basics

Git

Git is a extremely flexible, distributed version control system. GlusterFS' public mirrors are at http://git.gluster.com and at GitHub (https://github.com/gluster/glusterfs). The development repo is hosted inside Gerrit and every code merge is instantly replicated to the public mirrors.

A good introduction to Git can be found at http://www-cs-students.stanford.edu/~blynn/gitmagic/.

Gerrit

Gerrit is an excellent code review system which is developed with a git based workflow in mind. The GlusterFS project code review system is hosted at http://review.gluster.com. Gerrit works on "Change"s. A change is a set of modifications to various files in your repository to accomplish a task. It is essentially one large git commit with all the necessary changes which can be both built and tested.

Gerrit usage is described later in 'Review Process' section.

Jenkins

Jenkins is a Continuous Integration build system. Jenkins is hosted at http://build.gluster.com. Jenkins is configured to work with Gerrit by setting up hooks. Every "Change" which is pushed to Gerrit is automatically picked up by Jenkins, built and smoke tested. Output of all builds and tests can be viewed at http://build.gluster.com/job/glusterfs/.

Preparatory Setup

Here is a list of initial one-time steps before you can start hacking on code.

Register

Sign up for an account at http://review.gluster.com by clicking 'Register' on the right-hand top. You can use your gmail login as the openID identity.

Preferred email

On first login, add your @gluster.com email to your identity. You will have to click on the URL which is sent to your email and set up a proper Full Name. Make sure you set your @gluster.com email as your preferred email. This should be the email address from which all your code commits are associated.

Set Username

Select yourself a username.

Watch glusterfs

In Gerrit settings, watch the 'glusterfs' project. Tick on all the three (New Changes, All Comments, Submitted Changes) types of notifications.

Email filters

Set up a filter rule in your mail client to tag or classify mails with the header

 List-Id: <gerrit-glusterfs.review.gluster.com>

as mails originating from the review system.

SSH keys

Provide your SSH public key into Gerrit so that you can successfully access the development git repo as well as push changes for review/merge.

Clone a working tree

Get yourself a working tree by cloning the development repository from Gerrit

 sh$ git clone ssh://[username@]git.gluster.com/glusterfs.git glusterfs

Branching policy

This section describes both, the branching policies on the public repo as well as the suggested best-practice for local branching

Master/release branches

In glusterfs.git, the master branch is the forward development branch. This is where new features come in first. In fact this is where almsot every change (commit) comes in first. The master branch is always kept in a buildable state and smoke tests pass.

Release trains (3.1.z, 3.2.z, 3.2.z) each have a branch originating from master. Code freeze of each new release train is marked by the creation of the release-3.y branch. At this point no new features are added to the release-3.y branch. All fixes and commits first get into master. From there, only bug fixes get backported to the relevant release branches. From the release-3.y branch, actual release code snapshots (e.g. glusterfs-3.2.2 etc.) are tagged (git annotated tag with 'git tag -a') shipped as a tarball.

Personal per-task branches

As a best practice, it is recommended you perform all code changes for a task in a local branch in your working tree. The local branch should be created from the upstream branch to which you intend to submit the change. If you are submitting changes to master branch, first create a local task branch like this -

 sh$ git checkout master
 sh$ git branch bug-XYZ && git checkout bug-XYZ
 ... <hack, commit>

If you are backporting a fix to a release branch, or making a new change to a release branch, your commands would be slightly different. If you are checking out a release branch in your local working tree for the first time, make sure to set it up as a remote tracking branch like this -

 sh$ git checkout -b release-3.2 origin/release-3.2

The above step is not necessary to be repeated. In the future if you want to work to the release branch -

 sh$ git checkout release-3.2
 sh$ git branch bug-XYZ-release-3.2 && git checkout bug-XYZ-release-3.2
 ... <cherry-pick, hack, commit>

Commit policy

For a Gerrit based work flow, each commit should be an independent, buildable and testable change. Typically you would have a local branch per task, and most of the times that branch will have one commit.

If you have a second task at hand which depends on the changes of the first one, then technically you can have it as a separate commit on top of the first commit. But it is important that the first commit should be a testable change by itself (if not, it is an indication that the two commits are essentially part of a single change). Gerrit accommodates these situations by marking Change 1 as a "dependency" of Change 2 (there is a 'Dependencies' tab in the Change page in Gerrit) automatically when you push the changes for review from the same local branch.

You will need to sign-off your commit (git commit -s) before sending the patch for review. By signing off your patch, you agree to the terms listed under "Developer's Certificate of Origin" section in the CONTRIBUTING file available in the repository root.

Provide a meaningful commit message. Your commit message should be in the following format

• A short one line subject describing what the patch accomplishes
• An empty line following the subject
• Situation necessitating the patch
• Description of the code changes
• Reason for doing it this way (compared to others)
• Description of test cases

Review process

rfc.sh

After doing the local commit, it is time to submit the code for review. There is a script available inside glusterfs.git called rfc.sh. You can submit your changes for review by simply executing

 sh$ ./rfc.sh

This script does the following:

• The first time it is executed, it downloads a git hook from http://review.gluster.com/tools/hooks/commit-msg and sets it up locally to generate a Change-Id: tag in your commit message (if it was not already generated.)
• Rebase your commit against the latest upstream HEAD. This rebase also causes your commits to undergo massaging from the just downloaded commit-msg hook.
• Prompt for a Bug Id for each commit (if it was not already provded) and include it as a "BUG:" tag in the commit log. You can just hit <enter> at this prompt if your submission is purely for review purposes.
• Push the changes to review.gluster.com for review. If you had provided a bug id, it assigns the topic of the change as "bug-XYZ". If not it sets the topic as "rfc".

On a successful push, you will see a URL pointing to the change in review.gluster.com

Auto verification

The integration between Jenkins and Gerrit triggers an event in Jenkins on every push of changes, to pick up the change and run build and test on it. If the build and tests execute successfuly, Jenkins marks the change as '+1 Verified'.

It is important to note that Jenkins verification is only a generic verification of high level tests. More concentrated testing effort for the patch is necessary with manual verification.

If auto verification fails, it is a good reason to skip code review till a fixed change is pushed later. You can click on the build URL automatically put as a comment to inspect the reason for auto verification failure. In the Jenkins job page, you can click on the 'Console Output' link to see the exact point of failure.

Reviewing / Commenting

Code review with Gerrit is relatively easy compared to other available tools. Each change is presented as multiple files and each file can be reviewed in Side-by-Side mode. While reviewing it is possible to comment on each line by double-clicking on it and writing in your comments in the text box. Such in-line comments are saved as drafts, till you finally publish them as a Review from the 'Change page'.

There are many small and handy features in Gerrit, like 'starring' changes you are interested to follow, setting the amount of context to view in the side-by-side view page etc.

Incorporate, Amend, rfc.sh, Reverify

Code review comments are notified via email. After incorporating the changes in code, you can mark each of the inline comment as 'done' (optional). After all the changes to your local files, amend the previous commit with these changes with -

 sh$ git commit -a --amend

Push the amended commit by executing rfc.sh. If your previous push was an "rfc" push (i.e, without a Bug Id) you will be prompted for a Bug Id again. You can re-push an rfc change without any other code change too by giving a Bug Id.

On the new push, Jenkins will re-verify the new change (independent of what the verification result was for the previous push).

It is the Change-Id line in the commit log (which does not change) that associates the new push as an update for the old push (even though they had different commit ids) under the same Change. In the side-by-side view page, it is possible to set knobs in the 'Patch History' tab to view changes between patches as well. This is handy to inspect how review comments were incorporated.

If further changes are found necessary, comments can be made on the new patch as well, and the same cycle repeats.

If no further changes are necessary, the reviewer can mark the patch as reviewed with a certain score depending on the depth of review and confidence (+1 or +2). A -1 review indicates non-agreement for the change to get merged upstream.

Submission Qualifiers

For a change to get merged, there are two qualifiers which are enforced by the Gerrit system. They are - A change should have at least one '+2 Reviewed', and a change should have at least one '+1 Verified'. The project maintainer will merge the changes once a patch meets these qualifiers.