Let me explain what I mean by locality and repetition. The current crop of configuration management tools decouple the configuration from everything else even when it is somewhat detrimental to do so. One case where it is detrimental is in application configuration. I believe that application configuration should live in the same repository as the application itself instead of some other repository or a chef cookbook or a puppet module. Better yet the actual chef cookbook or puppet module should be alongside the application and the build system can figure out how to package the whole thing so that the deployment mechanism can be as simple as possible.
One problem with this set up is that it can lead to a lot of repetition because several packages are likely to use similar or even exactly the same chef cookbooks and puppet modules but instead of being in some central location these cookbooks and modules are now duplicated in each application repository. The instinctive thing to do in this case is to factor out the commonality and put it in its own repository but I think this is incorrect because it introduces non-locality. Now to understand how the application works across its entire lifecycle you have to chase down dependencies in some other place and sometimes it is not clear at all where this other place is because of the separation between operations and product development.
So what is the correct trade-off here? I think non-locality makes things more magical than they need to be and enforcing locality leads to unnecessary repetition.
5
Chef solves this problem by using dependency management (Berkshelf) and by applying cookbook patterns. Using the Environment cookbook pattern, you are encouraged to store your cookbook WITH your application (and this does NOT result in duplicated cookbooks, because any library / base / dependent cookbooks will be resolved with the dependency management tool.)
When it comes to application deployment, your cookbook should be able to search your binary repository, find the install package, download it, and install it. Doing this allows you to correctly separate the builds of your application and of your configuration automation.
More importantly, here’s what not to do:
- Build a docker image by hand and call it good “because Docker”. Doing this is just as bad as having magical VM templates (and of course, nobody at your organization will understand how they were built, and that makes them difficult to update, etc). If you’re going to use containerization, build the images with the automation.
- Store everything in git-submodules, or build the entire repository. Instead make sure you’re using dependency management and do well-versioned releases of all your software, including your configuration automation.
- Overuse dependency management tools and break your code out into a million layers of abstraction. You really don’t need a base, base, base cookbook. Just keep it simple with a few layers as necessary, refactor as you go.
- Pretend that your configuration management solution is an “install script”. It’s not, it’s an all-inclusive way to describe the state of your servers.
- Build your cookbook with your application in the same build process. You do not want to do new builds of a large application when all you need is just to package up the cookbook. Instead, builds should be able to happen in parallel, especially if you have a large, distributed team.
At least in the case of Puppet, you should Google the following:
- Puppetfile (like Gemfile but for Puppet)
- r10k
- “control repo”
- Hiera
The proper composition of a control repo to be consumed by r10k will solve 90% or better of the issues you’ve mentioned above. Your custom application can then be deployed with another component level Puppet module and be instantiated with all of the right data/config.
Other configmgmt tools have more or less similar tools and concepts but I cant pretend to be an expert on them.
As to Docker, you could make your deployment artifact an image built with Puppet rather than a package deployed by Puppet but you will eventually run into similar issues of providing correct settings for your environment and then you’ll be looking at having to add something like etcd or Consul or ZooKeeper to the mix and…yes…that’s also complicated.
Sorry I can’t provide more detailed examples. Mobile. 🙂