In the delivery of software, and within the sense of the deployment pipeline, where do the responsibilities of build tools, like Maven, end, and the responsibilities of CI start?
As a rough example of a problem that arises; should build tools have any responsibility to the configuration and execution of acceptance tests when they are further down the pipeline than actually building the artifact?
I’d like an answer that addresses in the sense of deployment lifecycle phases rather than in specifics, like my example. Although examples would help bolster the answer.
2
This comment sounds right.
In my opinion, as much of the build code as possible should be in the product’s internal build scripts, such as ant’s build.xml, maven’s pom.xml, etc. This serves 3 purposes:
- Versions the build process along with the product. If you fork a branch of your product in source control and make a change that requires build changes (like say you are adding a minify step to javascript files that was not there before), nothing outside of your source control needs to change.
- Ease of development and on-boarding. As a developer, it is great for productivity when I can simply checkout the code from source control and run a build command, be it
ant clean buildormvn packageor whatever. - Removes the dependence on CI for builds. You should use CI for builds, but if Jenkins is down for any reason, you don’t want to be unable to build. CI should be about automating an existing streamlined build process.
Ideally, a CI server should be configured to simply do this:
- Checkout
- Run a build command, preferably the same command developers run or perhaps a super-set of those commands.
- Archive the build to some official build archive location.
Where this gets less clear is with processes that occur at build time but are not strictly needed to build, such as unit and integration tests.
My preference would be to include unit tests in the core build. That is, you cannot get a build, even in dev, without the unit tests running and passing. Of course, that requires a huge amount of discipline and rarely is done. If you are lucky enough to have an environment that runs automated unit tests, including them in the core build seems like a good idea.
Integration tests, however, can take much longer, so running them any time you just want to compile code can be a huge productivity drain. Imagine if every compile -> test cycle requires an hour-long integration test suite. You would compensate by compiling less often which means less testing and naturally no good will come from that.
What I would do is define such functionality in build scripts within source control but not wire them into the core build. I don’t know Maven very well, but for ant, I would define a separate target, like ant integrationTests that you need to explicitly run. That could be configured in the CI system and that way, developers can choose if/when to run it themselves on their systems.
To put into a pipeline view, perhaps something like this:
Checkout -> Build -> Integration Tests -> System Tests -> Performance Tests -> Load Tests -> Archive Build -> Deploy
The implementation of each of those steps should live inside the product’s build scripts in source control (except for checkout, otherwise you have a chicken or egg problem) and the build process owns the decision of which step to call in what order. Certain steps may not be needed on every build. Load tests, for instance, maybe only run once release.
That is my 2 cents. I am by no means an authority on the subject, but I have seen builds done well and builds done poorly.