I’m currently in the process of learning Chef — the infrastructure automation framework by Opscode — and wanted to share some of the things I’ve picked up along the way. To give you a little bit of background, we’ve chosen Chef at work to gradually replace our deployment and configuration system, which is, unfortunately, entirely based on custom Debian packages (and plenty of magic). There’s still lots of work ahead of us and I’m far away from being a Chef expert, but I do feel comfortable enough to give you some tips on how to get started with Chef.
Learn Ruby (First)
The first thing people will usually tell you about Chef is that it allows you to write “cookbooks” with “recipes” defining how to configure a server to do X, with the standard example being an “Apache” cookbook which sets up the Apache web server.
Then you’ll probably hear that the whole system is based on Ruby. While Opscode claims that you don’t need to have experience with Ruby to write recipes (thanks to the straightforward recipe DSL), I still decided to learn Ruby first. In fact, I had wanted to learn this wonderful language for a while and Chef gave me another good reason. In hindsight, that was the right thing to do, in particular in regard to more advanced topics like automated cookbook testing.
Sooner or later, you need to dive into Ruby.
Vagrant and Chef Solo
Fast forward and several Ruby experiments later, I came back. By then, I had not only become a passable Ruby programmer, I also knew the best way to get started with Chef: Vagrant. (If you’ve been following this blog, you already know I have a thing for it.) Vagrant comes with provisioners for two flavors of Chef: Chef Solo and Chef Server.
- The Chef Server can be used to distribute cookbooks, manage and authenticate nodes, and to query infrastructure information. It is, however, completely optional and I will blissfully ignore it in this post. (The same goes for its counterpart: Chef Client)
- With Chef Solo, you can run cookbooks in the absence of a Chef Server. All you need are the cookbook files somewhere on your disk. Sounds simple enough? As you will see in a minute, it really is.
The Guts of a Cookbook
Before continuing with Vagrant and Chef Solo, I want to tell you a bit more about Cookbooks.
Cookbooks are the fundamental units of distribution in Chef. They encapsulate all the resources you need to automate your infrastructure and are easily sharable with other Chef users. -- Opscode
To put it simple, a cookbook is a bunch of files and directories. For someone new to Chef, the most important components to know are:
- Attributes define default values to be used by the cookbook, e.g. installation path or service port. Those values may be overridden by role-specific or node-specific values.
- Recipes specify (mostly) platform-independent Resources which are executed in the order defined to configure the node. For example, the
packageresource installs software packages,
remote_filedownloads a file, and
bashexecutes shell code.
- Files and Templates are transferred to specified paths on the node. While the former are static, the latter are powered by ERB — ideal to generate configuration files.
- Metadata describes the recipes, dependencies, supported platforms, etc.
I’ve recently written a Chef cookbook that installs Git from source and optionally configures it. Its directory structure is shown below. You can dig into the source code at GitHub.
. ├── attributes/ │ ├── config.rb │ └── source.rb ├── metadata.rb ├── README.md ├── recipes/ │ ├── config.rb │ └── source.rb └── templates/ └── default/ └── gitconfig.erb
Now enough with the theory. Let’s provision a virtual machine with Vagrant, Chef Solo, and some cookbooks! The result will be a great learning resource for Chef — a playground that you can use for further research and testing. (If you need an introduction to provisioning with Vagrant, I recommend reading my last blog post first. I won’t repeat the basics here.)
As a practical example, let’s use the Git cookbook I mentioned earlier. The cookbook’s
git::source recipe will compile and install Git on the target node. It depends on two other cookbooks:
- apt — updates the Debian package index for us (a sane thing to do for Debian/Ubuntu VMs)
- build-essential — installs everything required to compile C programs like Git (gcc, make, etc.)
To make a long story short, here is how to set things up:
# Create a new project folder $ mkdir cookbook-dev && cd cookbook-dev/ # Clone required cookbooks into "cookbooks" folder $ mkdir cookbooks $ git clone git://github.com/opscode-cookbooks/apt.git cookbooks/apt $ git clone git://github.com/opscode-cookbooks/build-essential.git cookbooks/build-essential $ git clone git://github.com/mlafeldt/git-cookbook.git cookbooks/git # Create Vagrantfile as seen below $ vim Vagrantfile
Vagrantfile tells Vagrant what to do:
Vagrant::Config.run do |config| # VM will be based on Ubuntu 12.04 (64 bit) config.vm.box = "precise64" config.vm.box_url = "http://files.vagrantup.com/precise64.box" # Configure the Chef Solo provisioner config.vm.provision :chef_solo do |chef| # Tell Vagrant where the cookbooks are located chef.cookbooks_path = "cookbooks" # Add recipes to be executed in given order chef.add_recipe "apt" # Shortcut for apt::default chef.add_recipe "git::source" # Will include build-essential::default end end
Now that we have everything in place, run
vagrant up to boot the VM and let Chef Solo do the work for us. As you will see in the log output, the cookbooks are executed in the order defined.
$ vagrant up ... [default] Booting VM... ... [default] Running provisioner: Vagrant::Provisioners::ChefSolo... [default] Generating chef JSON and uploading... [default] Running chef-solo... [Mon, 10 Sep 2012 14:51:25 +0000] INFO: *** Chef 0.10.10 *** ... [Mon, 10 Sep 2012 14:51:25 +0000] INFO: Run List expands to [apt, git::source] [Mon, 10 Sep 2012 14:51:25 +0000] INFO: Starting Chef Run for precise64 ... [Mon, 10 Sep 2012 14:53:49 +0000] INFO: bash[build and install git] ran successfully [Mon, 10 Sep 2012 14:53:49 +0000] INFO: Chef Run complete in 144.378525 seconds
The result speaks for itself:
$ vagrant ssh vagrant@precise64:~$ which git /usr/local/bin/git vagrant@precise64:~$ git --version git version 1.7.12
I’m aware that I really just scratched the surface of what you can do with Chef. Fortunately, there is a vital community around the project that has produced tons of information. Here are the sites I found most helpful while learning Chef:
- Opscode Chef Wiki — The official Chef documentation; comprehensive and really well organized. The first site you should visit for anything related to Chef.
- Opscode public cookbooks — For best practices, I always turn to production cookbooks. You can learn so much by reading the source code of tried and tested recipes. For more complex cookbooks, definitely check out apache2 and mysql.
- Vagrant provisioners — Explains the basics of provisioning with Vagrant as well as its different provisioners, incl. Chef Solo. (You might also read my last blog post which introduces Vagrant’s shell provisioner, along with provisioning 101.)