Why aren’t you using Vagrant?

last updated February 16, 2014

I’ll admit it. Six months ago, someone asked me if I was using Vagrant for dev setup. “Oh I usually prefer just to configure whatever I need on my machine.” I thought a vm would be overweight, clunky, and slow me down. I was wrong.

Very wrong.

All the wrong reasons not to.

Who really cares if your Redis instance or database is a few versions off from prod? That’s what staging is for. The CI build will catch those discrepancies. I’m fully capable of figuring out what requirements I need to install locally. I’ll just put those into the Readme. How hard can it be to stand up a few services and have them running in the background? Or maybe I’ll use foreman so they only run while I’m working. I wouldn’t want that Mongo DB process taking up 1MB and 0 CPU all the time. That would be silly.

These are all of the things I said to argue myself out of using Vagrant. But it only took one project to change my ideas around how to exploit virtual machines for development, and the potential to use them beyond.

A (purely) hypothetical scenario

Let’s say there’s some sort of distributed system composed of four or five components. Each of these are in their own repository. They share databases, a search engine, Redis instances, an old version of Java, and are taped together with a message bus. How do you on board someone to this scenario?

Simple.

1. Clone a single repository.
2. Run a single script to install Vagrant and virtual box.
3. Vagrant up.

HOLY !@#%#@% That’s it? It can be. Here’s some specifics on how to do something so silly.

Create a Vagrant machine

mkdir project
cd project
git init
vagrant init

So far so good. The Vagrantfile we created is full of helpful information. Get rid of everything for now. Leave only VAGRANT_API_VERSION and the Vagrant.configure block. I’m assuming some Ruby knowledge here, so I apologize if you’re coming from a different language.

Choosing a box

Vagrant has a slew of different virtual machine snapshots that are all ready to use. Choose the one that’s most like what you’re going to be using in production. This may even be the exact box you will use in production so choose carefully. I’m an Ubuntu guy, and I want it to have Chef preinstalled so I’m going to choose that box and add it to my Vagrant file with the following lines.

config.vm.box = "opscode-ubuntu-12.04_chef-11.4.0"
config.vm.box_url = "https://opscode-vm-bento.s3.amazonaws.com/vagrant/opscode_ubuntu-12.04_chef-11.4.0.box"

Now run vagrant up. The first install will have to download the image we just specified but once that’s done you should be able to vagrant ssh into that box. Huzzah. We’re up and running. Exit the vm and check in your Vagrant file. We’re ready to start doing cool things.

Provisioning

I mentioned Chef, but it is its own topic entirely. In fact, provisioning is an entire series of blog posts. I want to keep this as simple as possible and get you on the Vagrant bandwagon, so let’s stick to stock shell scripts. We will provision this box to run an Apache web server. While this could be an inline script, let’s create a separate file in ./scripts/provision/install_apache.sh and put a few lines to install and configure Apache.

#!/bin/bash
apt-get install -y apache2
cp /vagrant/scripts/provision/conf/my-site.conf /etc/apache2/sites-available/
ln -s /etc/apache2/sites-available /etc/apache2/sites-enabled
service restart apache2

Here we’re installing Apache, copying our site configuration into the available sites, linking it into enabled sites, and restarting apache. But you already knew that from reading the four lines, right? Worth noting is the lack of sudo. All scripts run as root unless told not to. Also notice the file lives in ./scripts/provision on our host, but we reference that at the absolute path /vagrant/scripts/provision as this script runs from within the guest VM. It’s important to be clear about which scripts run from within the VM and which run from the host machine, lest it become the traditional junk drawer.

Next we need to tell Vagrant to run this script when we provision the machine. Add these two lines to your Vagrantfile.

config.vm.network :forwarded_port, guest: 80, host: 8001
config.vm.provision :shell, path: "scripts/provision/install_apache.sh"

Then run vagrant reload --provision. Reload is literally restart, but the –provision flag tells Vagrant to run whatever provisioning may have been added since we first did Vagrant up. At this point you should be able to see the default Apache page at http://localhost:8001 since we forward the guest machine’s port 80 to 8001 on our host. It’s important to pick ports that won’t collide with other services.

Rinse and repeat this for each service you require installed on your machine, remembering to check in along the way.

Cloning repositories

Now we have the pain of cloning six repositories. If only there were some way…. OH YEAH COMPUTERS! Let’s create a script that clones each of the repositories into ./repositories. Before we do that let’s create that directory and add it to our .gitignore. Because each of the repositories there are repositories themselves, we do not want changes tracked in the Vagrant repo.

mkdir repositories
echo "repositories" >> .gitignore

And our cloning script, in ./scripts/provision/grab_projects.sh.

#!/bin/bash

needs_stash() {
    [[ $(git diff --shortstat 2> /dev/null | tail -n1) != "" ]] && echo "*"
}

git_grab(){
    DIR=/vagrant/repositories/`basename $1 .git`
    if [ ! -d $DIR ]; then
    git clone $1 $DIR
    else
      pushd $DIR
      if needs_stash; then
        git stash
        git pull --quiet --rebase
        git stash pop
      else
        git pull --quiet --rebase
      fi
      popd
    fi
}

git_grab git@github.com:project1
git_grab git@github.com:project2
git_grab git@github.com:project3
git_grab git@github.com:project4
git_grab git@github.com:project5
git_grab git@github.com:project6

The script is a little fancier than it needs to be for the purpose of this post. It clones or rebases depending on the state of the projects. Idempotency is important to consider when building provisioning scripts. Because the projects lives beyond the life of our ephemeral virtual machines and we might depend on running things like database migrations or starting services, we want to ensure we pull the latest and greatest. But that’s specific to how you work with git and irrelevant.

So let’s add what we need to our Vagrant file.

config.ssh.forward_agent = true

config.vm.provision :shell, inline: "apt-get install -y git-core"
config.vm.provision :shell, path: "scripts/provision/grab_projects.sh", privileged: false

Slow down, what’s going on here? Well first of all we’re cloning from Github. Assuming you have your ssh-keys configured for Github, it knows who you are but it does not know who the vagrant user on your VM is. In order to do that we must use ssh agent forwarding so the machine can assume your identity when authenticating from Github. If you’re on OSX, run ssh-add -K to permanently add your key to the agent. If you’re on Linux ssh-add will add it but there’s no -K option to persist it. I did this once before but gave up on day to day Linux. Please share how to do this in comments if you’re a Linux user.

Next we have to do something somewhat essential. Install git inside the VM. The commands are running from within the VM, so git must be there for our script to execute. We install this using an inline script because it doesn’t really deserve its own file.

Finally we run the grab_projects script. The only thing to take note of here is that we set the privileged flag. That’s because we want to run the script as the vagrant user, not root. Only the vagrant user gets your ssh key added to the key ring, so it’s required. We also don’t want the files owned by root inside the VM, that would be a pain.

Gateway drug to DevOps.

I really glossed over a lot of things here, but as you can see configuring your VM is really no more difficult than recording the steps you took to prepare your local machine in a script, then checking them in to share with your team. It’s even possible to split the dependencies out into a multi-machine vagrant file so services are isolated. Then you can switch from Virtual Box to AWS via the vagrant-aws plugin. Now you’re not just production like. You are production!

Of course shell scripts might not cut it, but you can easily swap in any of the available provisioners such as Chef, Puppet, Docker or whatever fancy tool you choose to cut your teeth on.

So I end with the question I started with. Why aren’t you using Vagrant? It’s cross platform, cross language, easy to use and awesomely powerful.

Tandem is software development companies in San Francisco with practice areas in custom mobile and web application development.