Vagrant on steroids

Source

This is our life when we are working on automating some really complicated machine building and provisioning procedures. I.e. while developing the playbooks in Ansible, or CookBooks in Chef or manifests in Puppet.

It is not easy to fail faster and fix early as the script might have to download all the dependencies again and again. Adding to all the odds, it could be that the dependencies are being downloaded on a low bandwidth.

Even though Vagrant makes bringing up VM and their management faster, the provisioning (using Ansible, Chef or Puppet or etc) might take inordinately long times when it involves, and it usually does involve downloading packages on the VMs. And it gets painful when you have to download a full stack of several libraries to test your VM.

To help overcome this issue, we can use the following to cache the dependencies and test the configuration scripts or recipes faster, the following should set you up (assuming you have vagrant already installed):-

1. Install vagrant-cachier plugin

vagrant plugin install vagrant-cachier

2. Install vagrant-proxy plugin

A Vagrant plugin that configures the virtual machine to use specified proxies.

vagrant plugin install vagrant-proxyconf

3. Install a proxy server

I am on mac and hence using squidman to install it use the below one line command

sudo brew cask install squidman

Configuring to use them

1. Configuring to use the vagrant-cachier.

Edit the Vagrantfile to include these line

 if Vagrant.has_plugin?("vagrant-cachier")
    config.cache.scope              = :box
    config.cache.synced_folder_opts = {
        type:          :nfs,
        mount_options: ['rw', 'vers=3', 'tcp', 'nolock']
    }
  end 

2. Configuring vagrant-proxyconf

Change the IP appropriately

if Vagrant.has_plugin?("vagrant-proxyconf")
# Start squid-man Mac OSX proxy on port 8081
# config.proxy.enabled = false
config.proxy.http     = "http://10.211.55.2:8081"
config.proxy.https    = "http://10.211.55.2:8081"
config.proxy.no_proxy = "localhost,127.0.0.1"
end 

3. Launch SquidMan and configure

As in the screenshots below. General tab, change the IP appropriately.

Clients tab add the following depending on your subnet

And start the Squidman

Confirming that it is all working

1. If you are on Ubuntu for example you would start seeing a lot of packages in the following folder
~/.vagrant.d/cache/parallels/ubuntu-14.04/apt/

2. Run the vagrant up and check squid's access logs. (Open Squidman and press Command+T and click on "Access log")

3. After these configurations have been put in place, Provisioning gets faster from the second time onwards. As for the first time, the dependencies get cached.

And for dependencies like JDK (which don't get cached due to https) you might want to create an image by snapshotting the VM with the OS and these dependencies. This will allow you to develop recipes you can skip their installations every time you run the palybooks or cookbooks


Hope this helps. (I am finally going to hit the Publish button :) after so many days of slacking on this. However do let me know if you would need clarity on anything)