Building BuildAPI Docker app

I am working through building up the BuildAPI Docker app that is going to live and work in Vagrant with the other apps. Currently I have things to the stage where buildapi is fully installed, and running but am having problems with kombu.

I have run into these issues before, when I was configuring kombu for the first time on my local machine. When I run docker build -t buildapi-app . and then docker run -i -p 5000:5000 -t buildapi-app the app starts up fine, but I see the following message about kombu:

localhost:buildapi-app jzeller$ docker run -i -p 5000:5000 -t buildapi-app
Starting subprocess with file monitor
Running reloading file monitor
Starting server in PID 7.
2014-04-16 05:51:00,389 WARNI [buildapi.lib.mq] [Dummy-2] Kombu connection failed ([Errno 111] Connection refused); waiting 2s
serving on 0.0.0.0:5000 view at http://127.0.0.1:5000
2014-04-16 05:51:02,396 WARNI [buildapi.lib.mq] [Dummy-2] Kombu connection failed ([Errno 111] Connection refused); waiting 4s
2014-04-16 05:51:06,407 WARNI [buildapi.lib.mq] [Dummy-2] Kombu connection failed ([Errno 111] Connection refused); waiting 6s
2014-04-16 05:51:12,419 WARNI [buildapi.lib.mq] [Dummy-2] Kombu connection failed ([Errno 111] Connection refused); waiting 8s

Here is the current Dockerfile for buildapi-app:

FROM ubuntu:precise
MAINTAINER Chirs Atlee <catlee@mozilla.com>
MAINTAINER John Zeller <johnlzeller@gmail.com>

# Install base dependencies
RUN apt-get -q update
RUN apt-get -y -q install python-dev mercurial sudo python-pip subversion wget curl make python-virtualenv libmysqlclient-dev sqlite3

# Install Python Google Visualizations 1.7.1
RUN pip install -e svn+http://google-visualization-python.googlecode.com/svn/trunk@18#egg=gviz_api.py-1.7.1-py2.7-dev_r18

# Install BuildAPI
RUN hg clone http://hg.mozilla.org/build/buildapi

# Install pip requirements with pip install
RUN curl http://python-distribute.org/distribute_setup.py | python; rm *.tar.gz;
RUN pip install -r buildapi/rtfd-requirements.txt
RUN pip install kombu==3.0.12 MySQL-python==1.2.3 python-memcached pytz==2011h

# Setup app
RUN cd buildapi; python setup.py develop; paster make-config buildapi config.ini
RUN cd buildapi; sed 's/buildapi.cache = redis:HOSTNAME:PORT/buildapi.cache = memcached:127.0.0.1:5001/' config.ini > tmp; rm config.ini; mv tmp config.ini

# Setup sqlite dbs
#RUN cd buildapi; sqlite3 statusdb.sqlite3 < statusdb_schema.sql
#RUN cd buildapi; sqlite3 schedulerdb.sqlite3 < schedulerdb_schema.sql
#RUN cd buildapi; sqlite3 buildapi.sqlite3 < buildapi_schema.sql

# Setup Kombu
RUN cd buildapi; sed 's/mq.kombu_url =/mq.kombu_url = localhost/' config.ini > tmp; rm config.ini; mv tmp config.ini

EXPOSE 5000

CMD [“paster”, “serve”, “–reload”, “-v”, “buildapi/config.ini”]

 

Further Steps with Vagrant

Today I spent a bunch of time looking into how to setup a vagrant instance, how to pack it with docker, how to run multiple docker containers in that vagrant instance and how to link them all together to play nicely. Throughout this process I have narrowed down that vagrant will be running hashicorp/precise64, and docker will have 5 apps, one each for buildapi, mysql , redis, rabbitmq and buildbot. Once I have linked all of this together properly, there will be a directory structure that looks a little like this:

./Vagrantfile

./redis-app/Dockerfile

./rabbitmq-app/Dockerfile

./mysql-app/Dockerfile

./buildbot-app/Dockerfile

./buildapi-app/Dockerfile

The Vagrantfile will load a base image of Ubuntu 12.04 Precise 64-bit, install docker (tip), build the images for buildapi (tip), mysql (tip), redis (v2.4.5), rabbitmq (tip) and buildbot (tip), expose all the necessary ports for each and link them properly, and then run each container as a daemon. Once this is all setup the user should be able to simply pull this repo (I am assuming this directory will become a repository on hg.mozilla.org/build), run "vagrant up" and then they will be able to see buildapi at 127.0.0.1:5000 and buildbot at 127.0.0.1:8501. The awesome added benefit to this is that files can be shared between your normal development environment and the VM that vagrant started up, simply by placing and editing files in the base directory where Vagrantfile exists! This will make development of these apps a lot smoother for our community members, with the super simple push button command line interface that vagrant provides for getting these VMs started up. Additionally, once this works it can easily be expanded with more and more apps as we see fit to add them to this Vagrant instance! :)