Continuous Integration – Don’t just kick the can further down the road

Following my previous blog on Extreme Programming, I wanted to talk about another XP practice, Continuous Integration or CI as it is popularly known. CI is one of the most valuable practises you can bring to a development team and because of that it has become almost a mainstream activity with nearly every team I meet practicing it. Or at least claiming to practice it, as there are some very common misconceptions about CI that I’d like to discuss here.

In my last post, you may remember we talked about the definition of Working Software coming from XP: “Software that passes customer-defined tests in a production (or production-like) environment.” Known in Scrum as “Done”. If your work is not passing the acceptance tests you agreed with your customer in the production environment, or an environment that is as identical to the production environment as to make no difference, there is a risk that it will come back to you. In other words, if your work does not meet this criteria, it is not yet completely integrated.

This brings me to a conversation, a very common conversation, that happens in pretty much every Certified Scrum Developer course I run. The conversation starts with me asking the class how many of them are practising Continuous Integration. Most participants raise their hands but when I question them about how often they integrate their code into the trunk, the answer is often, “we don’t integrate into our mainline, we use feature branches instead.”

They usually go on to inform me that having half-developed features on the mainline makes it difficult to release and so the use of feature branches is necessary to overcome the problem of releasing.

At this point I emphasise to the class that integration only takes place once all of the required branches are completed and merged together. Simply by definition, Continuous Integration means just that; to integrate continuously. The purpose of Continuous Integration is to provide rapid feedback about the state of the system after every single change has been made. The whole system, that is. Not just the feature you’re working on. It can only happen when all the pieces of the puzzle have been fitted together. A true continuous approach, should involve integrating and testing everything probably several times a day. Even as frequently as every half-hour.

As Feature Branches are typically developed over relatively long periods of time, when the time comes to integrate the feature branches with the main trunk they can throw up a large number of bugs and errors. This in turn, means having to go back and revisit coding decisions you made a very long time ago; months or even years in the most extreme cases. Therefore, although they may solve the problem of releasing while some features are still only half developed, the use of Feature Branches developed in isolation of one another and integrated infrequently, hinders the objectives of Continuous Integration.

It reminds me of when I was a young boy and loved playing football (soccer to any US friends reading this.) One of my favourite forms of practice was to kick an empty drinks can along the road in front of me on the way home from school. Most days I was committed to the task of keeping the can under control, merely nudging it a few inches with one foot before gently tapping it with the other, never letting it stray far from the direction I wanted to go. Other days, I was less attentive and would give the can a massive kick, sending it a long way in front of me and allowing me the freedom to stroll without needing to pay attention to the can. This often caused the can to stray from the intended path. I’ve not researched it but I imagine there is a relationship between how hard I kicked the can and how far from the path the can ended up. The further from the path the can was, the more work I had to do to get it back on track. On those days I cared little about the remedial work of getting the can back on track because it meant I could walk without the bother of looking after the can.

Fully integrating frequently can seem like additional work. It does identify defects and merge errors, forcing you to spend time fixing them when you thought you’d finished and all you really want to do is move on to the next piece of work. A positive side-effect of that is that you get practice at fixing these errors and as you become more familiar with these conflicts, you become quicker at fixing them. The more you do it, the better you get at it.

This is a great blog on how Moonpig transitioned from Feature Branches to single trunk development earlier this year. Here’s what they say, “Long running feature branches inevitably result in messy merges, and avoiding those is a great benefit of trunk based. Whilst merge conflicts do not entirely go away, they are less frequent and far less difficult to unpick. Above all, it’s much, much faster!”

As Continuous Integration gains wider exposure, more people and organisations are keen to adopt this practice. However, simply using a source control management system doesn’t mean you are doing CI. If you’re using Feature Branches, it probably means you’re not. All you’re doing is kicking the can further down the road.

If you want to establish a stronger understanding of how to successfully implement Continuous Integration, why not join us on one of our Certified Scrum Developer dates in the New Year. They are available in either Java or .NET.

Certified Scrum Developer (Java)
Certified Scrum Developer (.NET)