Today I spent a bunch of time looking into how to setup a vagrant instance, how to pack it with docker, how to run multiple docker containers in that vagrant instance and how to link them all together to play nicely. Throughout this process I have narrowed down that vagrant will be running hashicorp/precise64, and docker will have 5 apps, one each for buildapi, mysql , redis, rabbitmq and buildbot. Once I have linked all of this together properly, there will be a directory structure that looks a little like this:
The Vagrantfile will load a base image of Ubuntu 12.04 Precise 64-bit, install docker (tip), build the images for buildapi (tip), mysql (tip), redis (v2.4.5), rabbitmq (tip) and buildbot (tip), expose all the necessary ports for each and link them properly, and then run each container as a daemon. Once this is all setup the user should be able to simply pull this repo (I am assuming this directory will become a repository on hg.mozilla.org/build), run "vagrant up" and then they will be able to see buildapi at 127.0.0.1:5000 and buildbot at 127.0.0.1:8501. The awesome added benefit to this is that files can be shared between your normal development environment and the VM that vagrant started up, simply by placing and editing files in the base directory where Vagrantfile exists! This will make development of these apps a lot smoother for our community members, with the super simple push button command line interface that vagrant provides for getting these VMs started up. Additionally, once this works it can easily be expanded with more and more apps as we see fit to add them to this Vagrant instance!
I am working with Vagrant and Docker for the first time, and it is awesome! We are working towards capturing all of our "how to install" knowledge for buildbot, buildapi and redis in code by making a set of docker containers that can play well together inside of Vagrant. In this setup, we are assuming that the user has Ubuntu (or equivalent linux) in a virtualbox running. Once that 1 VM exists we are to have separate docker instances per app (buildbot, buildapi, redis, etc.). The benefit of connecting these docker instances inside vagrant to play together, is that it lets any combo of buildbot, buildapi standalone, or "system" to be run. This also allows more modules to be added later.
So far I have installed Vagrant, Docker and Boot2Docker, worked my way through the Boot2Docker/Docker tutorials (including a sweet hello world app example from David Dias), and I have begun working my way through the Vagrant tutorials. Other than all that prep, I have also begun writing up the Dockerfiles needed to setup each of our apps. I think we will need one each for:
I got most of the way through the BuildAPI and RabbitMQ Dockerfile's before I got to a point where I needed to get Vagrant installed and figured out so that I could also then create a Vagrantfile to go along with this entire setup.
Once I have completed this entire setup, you will be able to get a buildapi, buildbot, redis, rabbitmq, and selfserve system up and running simply by downloading the Vagrantfile and running 'vagrant up'. How cool is that?!
More updates on this next week!
Bug 793989 requests the ability of buildapi's self-serve to be able to request arbitrary builds/tests on a given push, and this is now possible! If you are interested in triggering an arbitrary build/test you can start with this simple python script to get you started.