Multi-stage continuous integration is a software development technique intended to achieve highly integrated parallel development activity while reducing the scope of integration problems.[1]
Multi-stage continuous integration takes advantage of a basic unifying pattern of software development: software moves in stages from a state of immaturity to a state of maturity, and the work is broken down into logical units performed by interdependent teams that integrate the different parts together over time. What changes from shop to shop is the number of stages, the number and size of teams, and the structure of the team interdependencies.
Multi-stage continuous integration is an expansion upon continuous integration, it presumes that you are already following those recommended practices.
The larger and/or more complex the project, the higher the chance that the project becomes unstable. Alerts and broken builds increase as the project grows. Progress decreases and the mainline gets increasingly unstable. The risk of build failure increases exponentially as the number and locations of developers grow.[2]
Each developer works on their own task. As they make changes, continuous integration is done against that team's branch. If it does not succeed, then that developer (possibly with help from her teammates) fixes the branch. When there is a problem, only that team is affected, not the whole development effort. This is similar to how stopping the line works in a modern lean manufacturing facility. If someone on the line pulls the "stop the line" cord, it only affects a segment of the line, not the whole line.
It is note-worthy that in recent years the "topic" or "feature" branch model has gained in popularity over the team based branch model. See for example the popular Git-Flow branching model [3]
On a frequent basis, the team will decide to go to the second phase: integration with the mainline. In this phase, the team does the same thing that an individual would do in the case of mainline development. The team's branch must have all changes from the mainline merged in (the equivalent of a workspace update), there must be a successful build and all tests must pass. Integrating with the mainline will be easier than usual because only pre-integrated features will be in it, not features in-process. Then, the team's changes are merged into the mainline which will trigger a build and test cycle on the mainline. If that passes, then the team goes back to the first phase where individual developers work on their own tasks. Otherwise, the team works on getting the mainline working again, just as though they were an individual working on mainline.
Changes propagate as rapidly as possible, stopping only when there is a problem. Ideally, changes make it to the main integration area just as frequently as when doing mainline development. The difference is that fewer problems make it all the way to the main integration area. Multi-stage continuous integration allows for a high degree of integration to occur in parallel while vastly reducing the scope of integration problems.[4]
For multi-stage continuous integration, each team must have its own branch.
Multi-stage continuous integration has many advantages: [citation needed]
Tools that support multi-stage continuous integration include: