Amplify your Bamboo workflow with git

January 20th 2015 Steve Smith in Git, Stash, Bamboo, CD, CI

Back in December I did a webinar on some of the advantages of using git with your Bamboo pipeline, leveraging some of the Bamboo and Stash integrations to create a feed-back loop of quality control. The transcript for this webinar is now available below the fold...

Amplify your Bamboo workflow with git

Hello and welcome to this webinar, "Amplifying Your Bamboo Workflow With Git." This webinar will hopefully show you a few cool tricks you can do with Bamboo and Stash to really enhance your productivity and code quality, while also making your life a bit easier as well.

But first, I should probably introduce myself. My name is Steve Smith, I've been at Atlassian for over eight years now. Back in the day, I was the original company's sysadmin at one of the earlier Sydney offices. But over the last five years I've been on the internal systems team.

The internal systems team built the Atlassian ordering systems. These are the financial systems that you purchase Atlassian tools through. Obviously, this has some strong requirements for quality and safety and this does have some relevance to some of the things I'm going to talk about a bit further on.

However, more recently, I moved to the Atlassian Amsterdam office in the Netherlands. This also has some relevance to leveraging Stash and Bamboo, for reasons I'll touch on during the talk.

Git workflows

Branching workflow

But before we start, we should probably have a look at the recommended workflows for Git. It has a lot of relevance to what I'm going to talk about and enables some of the tips and tricks that I'm about to discuss.

As you may know, Git enables some workflows that are somewhat different to workflows for more traditional version-control systems, such as Subversion or CVS. The key difference, when developing with Git, is that branching and merging is safer and faster than with older version-control systems, such as Subversion or CVS.

This allows us to isolate work on individual features, or bug fixes, while other work continues in parallel. We only merge the work when it is completed and has been reviewed by other members of the team, something I will discuss in detail later on.

While we are doing this branch-based workflow, it ought not break anything when we do merge back to the master branch. We should ensure that all tests are passing. To do this, we run all tests required on the master branch on the feature branch as well. This ensures the master branch constantly remains in a releasable state.

This, by the way, is one of the central points of the continuous delivery workflow. By making sure that the stable branch is always in a releasable state, you enable a lot of powerful new workflows, such as continuous delivering and continuous deployment.

But not everything can be detected by tests. Certain classes of problems in the code can be missed, either because tests have not been provided or because they entailed judgement calls by humans. Some things just require human input to make the best decision. The final stage of our future branch is to create a pull request and invite other team members to review our changes and check for issues that may not show up in tests.

Code reviews and pull requests have a lot of benefits that we're not going to be able to rehash here. There are a number of blog posts and articles on Atlassian and other sites that give you an overview of the key points and the best, most effective way to perform code reviews and pull requests.

In particular, I recently wrote a retrospective blog posts about how the internal systems team adopted pull requests organically as a method of gaining peace of mind when we were modifying our financial systems.

So we've seen that each branch and merge workflow enables lots of new opportunities to inspect and enhance our code quality as we are developing. But personally, I'm lazy. I want our tools to do as much possible for us. Bamboo and Stash maintain a huge amount of information about ongoing code activity and the quality of that code. We would like to leverage this to assist us as much as possible to maintain a high-quality code base at all time.

So how can Stash and Bamboo assist with this workflow? This is where a build and commit feedback would be useful. If we can share information between our code repository and our build server, we can enable some powerful interactions that make our lives easier as well.

Feeding forward

Branchin workflow

The first part of this Stash/Bamboo interaction I want to look at is feeding forward. Feeding forward is where one of the tools in our pipeline provides information to the tools further down the chain to help them make intelligent decisions about the work required of them. In the case of Bamboo and Stash, Stash is able to push information to Bamboo about changes that occur within the Git repository as they happen in real time. Bamboo can then use this information to automatically perform actions on our behalf.

Lets have a look at some of the actions that can be performed that will help you out on a day-to-day basis. The most powerful feature enabled by forward integration is that Bamboo can be informed about the creation and deletion of branches inside your Git repository. This enables Bamboo to create and remove branch plans as you branch and merge your features inside Git. If you are not familiar with them, branch plans are copies of old build tests and other quality-assurance metrics that are run against the master branch.

These are copied to a new plan for that feature branch, and they are then run on every commit. This enables the branch and test workflow that I talked about at the beginning. By running all the quality-control tests against the feature branch before we perform the merge back to the master branch, we are ensuring that when we do so the master branch remains in a releasable state.

This is absolutely core to continuous delivery and continuous deployment. In many cases, your master branch may actually trigger a release or a deployment to a production server at the time of merge, so it's absolutely critical that your master branch must remain in a pristine state at all times.

Just to give you a quick overview of how this looks inside Bamboo, here's some real world build results from a branch I was working on for our ordering system. As you can see, the branch plan looks much like a normal Bamboo build plan result.

It shows the commits that have been occurring on the branch. The results of the tests are run against that branch. But the major difference is that all the builds and tests are being run against my branch, rather than the master branch.

But other than that, it is largely the same as if you are running a simple, single-branch build plan. But to further enhance this feature, Stash will also inform Bamboo when commits occur inside the Git repository.

This allows Bamboo to trigger new builds whenever you commit to your feature branch, thereby speeding up your development test feedback loop. Instead of waiting for your tests to be triggered in Bamboo or manually triggering it for yourself, Stash will immediately inform Bamboo that a commit has occurred the moment you push to it.

This greatly speeds up your build test feedback loop and provides an improved productivity. This is particularly effective if you are using a continuous integration pipeline that runs your light, fast tests at the start of your pipeline and then continuously increases the complexity of your tests as you go down the line, up until full-acceptance testing at the end which may run on the simulacrums of real world systems.

By having the light tests at the start, you will often spot build and test issues early on in the pipeline. If your builds are triggered the moment you push to your Git repository, you will get this feedback much earlier in that feedback loop. This is important, as it speeds up your build test cycle greatly.

Feed back

Branchin workflow

More powerful though is the feedback portion of the feedback loop. This enables some of my favorite integration features between these two tools. I am someone who genuinely uses these tools on a day-to-day basis and discovering some of these build integrations are a real fun part of my working day sometimes.

What I mean by feedback is that, as well as receiving information from Stash, Bamboo can push information about the build and the test results back into Stash. Stash can then use this information to help developers make informed decisions about the quality of the code and even to prevent problems before they actually happen.

To illustrate this second part, I should give you a little bit more background. As I mentioned, until recently, I worked as developer on our ordering systems, but although most of my team is based out of Australia, I work out of the Amsterdam office which is an 11-hour time difference at this time of year.

This often meant that, by the time I started work in the morning, all the other developers were in the process of committing their changes for the day and then going home. Often, I would come in in the morning, create a branch for a new feature I was going to work on that day and then start making commits.

But sometimes I would find that my branch would immediately start failing builds in some weird place that had nothing to do with what I was changing at that time. Sometimes, I would actually spend hours tracking down the bug, discovering that the problem had nothing to do with the changes I have made.

This is in spite of the fact that master branch builds were green when I went and checked them. What was happening here is that the developer may sometimes make a change directly onto the master branch.

You know we are not supposed to, but it does happen sometimes. They may also merge a branch locally themselves and push it up -- and then push it without testing, more importantly. There may even be a problem with a tricky merge that introduces an error. Although, frankly, this is exceedingly rare inside Git.

It turns out, the change the developers made now breaks the build. It's fine. Luckily, Bamboo would immediately run the test for the master and alert that the build has failed. This is why we run continuous integration servers, to rapidly find such problems and feed them back to the developer to make the change.

No problem. The developer commits the fixes for this. They wait. The build is green again. All is well and good. Developer heads home for the night.

This temporary breakage could have some knock-on effects. Imagine that, while this is going on, another developer -- say one in Amsterdam -- starts to work to fix a bug in the data access layer, for example.

This developer does the right thing and creates a branch for his work, but once they start committing this branch and then push it up to the main Git repository which, in turn, triggers the builds. They see that builds immediately start failing. Unknown to them, they've branched from a bad point. They will then possibly think that they have introduced an error and spend hours tracking down a bug that really has nothing to do with them.

How can Stash help with this? If you create your branches from within Stash directly, via the web UI, we can avoid this situation. If Bamboo is configured to feed the build states back into Stash, Stash can warn the developer at the time of creating a branch that they are likely to encounter issues by branching at this point.

This requires you delegate branch creation to Stash, which is not to everyone's taste but it's often worth it to avoid these sorts of complicated issues -- particularly when a lot of merging and branching is going on at the same time.

It's also worth noting in passing that you can even create branches in Stash, from within JIRA, if you've enabled certain integrations there. This will also take care of your branch naming for you, which cuts out a whole other set of problems that you can be introduced by having typos in your branch names.

This is a quite complex subject in itself. I've really only touched on some of the really cool stuff that can be done there. If you are interested in learning about this really useful technique, my colleague Tim Pettersen recently did a webinar about this and a bunch of other cool integrations between Stash and JIRA. The webinar recording is available on the Atlassian YouTube channel. Go check that out, it is well worth the time. There's a lot of really cool stuff in there.

Pull request enforcement

Branchin workflow

I have saved my favorite use of feedback until last. I have talked a bit at the start about how important code reviews and pull requests are. They promote code quality and code knowledge within the team and can help spot problems in the code that are often hard to test for.

I particularly like pull requests, as they give me peace of mind when working on highly-critical code. I wrote a longer blog post about this recently and, if you wish, you can go check it out in blogs.atlassian.com.

As I mentioned, the internal system team work on the ordering systems for Atlassian. Any major errors introduced into this code could potentially cost the company a lots of money. We process the actual financial transactions.

So whenever I worked on areas of the code related to financial transactions or to business logic, I would always ensure that I got input from other members of the team who knew more about these areas than me.

My strengths are really in the technical area, and when it comes to business and finance logic, I am not as strong. However, I know there are certain people on the team who are good at that and have lot more domain knowledge in this area then me.

I will often pull them into pull requests and, by having them check my logic, I will often feel lot happier about the changes that I was making at that time.

However, by their nature, pull requests and code reviews are more work-intensive than code-based testing, and we want to ensure that this time is well spent by the developers who are being pulled in and are asked to check other people's code.

We also want to leverage information from the build system to ensure code quality. Ideally, we want to use the information from the builds that have been occurring on the branch to help define a policy on what standards must be meet before we merge to the master branch.

Stash provides this ability to define policy. Within each repository, we can set a number of parameters that must be met before a merge can be performed within that repository.

One of these is that we can require a certain number of reviewers to approve the pull request before it is merged. How many reviews this is is up to you. It depends a lot on your team resources and the type of code you are working on, and also your team structure. By ensuring that all code is seen by other developers before it enters the wider code base, you can ensure that subtle issues are avoided and knowledge is passed around the team as well.

Stash can even recommend reviewers to you at the time of raising the pull request by looking into the history of the code that you have modified, seeing who else has made changes there and making recommendations on who is the best person to be knowledgeable about this code base.

More relevant to our discussion is that we can require that the builds must be passing before the merge can be completed. This gives the team reassurance that they are not going to merge broken branches back into the code, code base.

It can also inform them during the code review that there may be problematic areas that have been missed or areas that they can possibly focus on. Here, I am showing how it looks in practice.

This is, again, a real example from pull request from the ordering systems I have been talking about. In this case, we can see that two reviewers have agreed that the pull request is good in principle. However, Bamboo has informed Stash that the build has failed some tests.

These broken builds prevent the merge from occurring until these tests have been fixed. Obviously, this is exceedingly powerful and prevents any breakage on the branches being merged into.

Conclusion

From these real world examples, we can see that having your build server and your Git repository communicate creates opportunities to really improve code quality and eliminate wasted time due to temporary breakages and failing tests that might otherwise be missed.

One thing that I often say is that, ultimately, software is written by humans for humans. We must include humans in the process of quality control, but if we can leverage the technology and information we already have available, we can reduce the burden on humans involved and better safeguard the quality of the code.

Thank you for attending this webinar on Git and Bamboo integrations. Hopefully, it has been useful to you. Of course, I will be hanging around after this to answer some of your questions online. We also have some more webinars coming up covering other areas of Agile and quality assurance. Do keep an eye out for those.

Once again, thank you for listening to this. Hopefully, it has been useful. We'll show some cool, interesting integrations that can make your life easier and generally just useful on day-to-day basis. Thank you!