Test-kitchen is an end-to-end testing framework for chef cookbooks. It manages VMs, runs your cookbook under test and then verifies the cookbook brought about the changes you expected.
Test-kitchen is moving quickly and documentation is sparse, so I found it pretty hard getting started with test-kitchen, vagrant-lxc and chef-zero. After lots of reading I finally realised the following setup.
VM Management
Test-kitchen has a plugin architecture that supports different virtualization backends. One of them is the kitchen-vagrant backend. It uses vagrant, which I think of as an abstraction layer on top of the actual virtualisation technique. As some of my colleagues are on Mac OS X, this should make it a little simpler for them.
To install vagrant, you have to download a package from http://downloads.vagrantup.com/ and install it manually.
By default, vagrant uses VirtualBox as virtualization backend. This is a pretty decent default for a cross-platform tool but as such, it is not the optimal solution for my use case.
Doing end-to-end testing with virtual machines will be slow as hell, even more so when you are used to unit testing where runtimes are measured in milliseconds. The fastest virtualization technique on Linux is, to the best of my knowledge, Linux Containers (lxc).
Luckily vagrant also has a plugin architecture, so switching to lxc is mostly a matter of installing prerequisite packages and the lxc plugin.
$ sudo apt-get install lxc lxc-templates cgroup-lite redir $ vagrant plugin install vagrant-lxc Fábio Rehm, the author of vagrant-lxc, has put together a blog post, explaining how to setup an lxc base box. At the end of this post he mentions to add your own stuff to the base container. For our intended setup, we need to install a few packages.
Log into the container with user and password vagrant, then do this:
$ sudo apt-get -y install curl build-essential $ cd /tmp && curl -O https://opscode-omnibus-packages.s3.amazonaws.com/ubuntu/12.04/x86_64/chef_11.6.0-1.ubuntu.12.04_amd64.deb $ sudo dpkg -i chef_11.6.0-1.ubuntu.12.04_amd64.deb && rm chef_11.6.0-1.ubuntu.12.04_amd64.deb $ sudo apt-get clean $ sudo /opt/chef/embedded/bin/gem install chef-zero knife-essentials $ sudo halt Installing chef and chef-zero into the base box speeds up test-kitchen runs considerably. If you don't mind the time installing those every test run, feel free to leave them out of the base box.
Finally create the base box according to Fábio's blog post. I extracted this shell script from his post.
Installing test-kitchen
I manage all my gems with bundler, so the first step to installing test-kitchen is to setup a Gemfile in the chef repository's root.
source 'https://rubygems.org' gem 'berkshelf', '~> 2.0.0' gem 'chef', '~> 11.6.0' gem 'chef-zero' gem 'json', '1.7.7' # needed for conflict resolution gem 'kitchen-vagrant' gem 'test-kitchen', '1.0.0.beta.3' We will use berkshelf to manage cookbook dependencies (see below). We have to list chef and chef-zero here (although we already installed them above), so that test-kitchen will be able to load them.
As described in the comment, the explicit json dependency is required for conflict resolution. Otherwise bundler will fail with varying errors.
Running bundle install should install all gems without errors.
Using test-kitchen
To get started we initialise an empty cookbook.
$ mkdir cookbooks $ knife cookbook create my_sample_cookbook -o cookbooks $ cd cookbooks/my_sample and create a config file for test-kitchen in the cookbook's directory.
Let's have a look at some of the specifics of this file.
driver_plugin: vagrant provisioner: chef_zero First we setup our toolchain with vagrant and chef-zero.
driver_config: http_proxy: http://10.0.3.1:8123 https_proxy: https://10.0.3.1:8123 Using an http proxy, is a nice performance optimisation. Just run a local proxy like and make sure it listens on the lxc interface.
platforms: - name: ubuntu-12.04 In the platforms section we setup the different environments, test-kitchen will use to test our cookbooks.
suites: - name: default data_bags_path: ../../data_bags Finally, the suites define different test suites for the cookbook. For instance, you could have different test suites for client and server recipes.
By default, chef-zero uses roles, data bags and nodes from your cookbook, e.g. cookbooks/my_sample/data_bags. You can override these paths with roles_path, data_bags_path, and nodes_path if you prefer putting test data elsewhere or even using production data for your tests.
To run test-kitchen, call kitchen test and see it iterate over all platforms and suites. Nice!
Adding Tests
Depending on your preferences there are different ways of writing tests for test-kitchen. All of them (AFAIK) are based on minitest. I am used to rspec, so I prefer using minitest/spec.
If you google around, you can find different ways of providing tests to test-kitchen that work in one or another scenario. Here, I'll describe the simplest way I could find for making it work with minitest/spec.
First we create a Berksfile to manage cookbook dependencies.
metadata cookbook 'minitest-handler', github: 'btm/minitest-handler-cookbook' With the metadata option we delegate dependency specification to the cookbook's metadata.rb file. This way we don't duplicate that information. In the Berkfsfile we only add test-related cookbooks explicitly.
There were a few important bugfixes for the minitest-handler cookbook, but no new release yet, so we have to use the github master.
Berkshelf will automatically install all necessary cookbooks in our VMs.
Now that we have the testing framework installed, it's time to write an actual test. The easiest way is to put the test file in the cookbook's files/default/test folder. Other people use specific test-cookbooks but the reason for doing so is beyond me.
In my sample test I utilize chef's data bag search, which only works after including the proper dsl module. Expect a similar setup for anything non-trivial.
With the tests in place, running kitchen test will hopefully fail and only succeed after you have added a proper recipe to the cookbook.
If you have any improvements to this setup, let us know in the comments.
Happy testing!
No comments:
Post a Comment