Vagrant can now run BuildAPI and RabbitMQ apps

Continuing on from my previous post, I verified that buildapi and selfserve-agent are truly connected and able to exchange over the amqp, and that the entire buildapi application is running well by running similar procedures that work in my local setup.

Once I did that I updated the Vagrantfile to forward the vagrant port 8888 to the host port 8888, and to build and start the rabbitmq-app and buildapi-app. In the wild, the Vagrantfile will not be having docker build the docker images, but rather it will pull them from Mozilla's docker repository, which will be a much faster process. As it stands, running vagrant up from scratch the first time will take about 10-15 minutes to launch.

Here's how you can NOW run a fully functional BuildAPI app locally with a single command :)

  1. hg clone http://hg.mozilla.org/users/jozeller_mozilla.com/vagrant-docker-setup
  2. cd vagrant-docker-setup
  3. vagrant up
 

MySQL databases are all setup in BuildAPI-app docker container!

As I stated in the previous post, the next step here was to setup databases. I spent time attempting to have sqlite work in this situation, but ran into issues with buildapi connecting to the sqlite databases. Rather than chase that rabbithole, I doublechecked the configuration in production buildapi and was reminded by the configs that production is running mysql. So I went ahead and did so. This setup required adding the following to the Dockerfile:

RUN apt-get install -y mysql-server

RUN chown mysql.mysql /var/run/mysqld/

RUN mysql_install_db # Installs mysql database schemas

RUN /usr/bin/mysqld_safe &

After this, everything was peachy except for the sql schemas available in the current buildapi repo. Those schemas are for sqlite, so I dumped my own mysql schemas for use here, and loaded them with the following commands:

mysql < status_schema.mysql

mysql < scheduler_schema.mysql

I went ahead and submitted a patch to add the mysql specific schemas to the buildapi repo in Bug 1007994, but for now I added the schemas in with the files in the buildapi-app directory.

I uploaded the current contents of the buildapi-app docker container and it launches with schemas all loaded and running well.

I am still having some issues verifying that selfserve-agent can execute commands from data sent to it over the amqp by buildapi. Further testing is needed to fix this issue. I am currently getting 404 error with my tests, but that might be a peripheral problem rather than selfserve-agent not getting data from the amqp.

Left to do on buildapi-app is to:

  • Test that buildapi and selfserve-agent are truly connected and able to exchange over the amqp
  • Test the entire buildapi application by running similar procedures that should work in my local setup

Links I found useful for this:

  • http://ijonas.com/devops-2/building-a-docker-based-mysql-server/

 

BuildAPI-app is almost up!

I am very close to having the buildapi-app docker container working completely. I left off last not having selfserve-agent setup, and having a kombu error.

In order to setup selfserve-agent properly, I had to include a selfserve-agent.ini file in the base of the docker file to be used by selfserve-agent.py when called with: python buildapi/buildapi/scripts/selfserve-agent.py -w; Additionally, I included a simple bash script to ensure that the container is able to launch both processes side by side without blocking one another.

The error I was having with kombu was because I did not have rabbitmq-app running. Kombu is used (as carrot was before) to make a connection to the amqp that rabbitmq sets up as an mq. After getting rabbitmq-app up, it needed to be linked with buildapi-app, and once it was it became clear that localhost was not the proper host for buildapi or selfserve-agent to attempt to find the amqp. When docker links containers, it allocates all the ports and IPs for them. It makes these new connections available to you in the form of environment variables. Once I had the 2 apps up and linked by running:

docker run -d -p 5672:5672 -p 15672:15672 -p 4369:4369 -name rabbitmq rabbitmq-app

docker run -t -i -p 8888:8888 -link rabbitmq:mq -name buildapi buildapi-app /bin/bash     # bash so that I can play with the variables

Then I was able to run env and see the environment variables that docker setup:

HOSTNAME=ee13bea5d0db
TERM=xterm
MQ_PORT_4369_TCP_ADDR=172.17.0.2
MQ_PORT_5672_TCP=tcp://172.17.0.2:5672
MQ_PORT_5672_TCP_PORT=5672
MQ_PORT_5672_TCP_ADDR=172.17.0.2
MQ_PORT_15672_TCP_PORT=15672
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
MQ_PORT_4369_TCP_PORT=4369
PWD=/
MQ_PORT_15672_TCP_ADDR=172.17.0.2
SHLVL=1
HOME=/
MQ_PORT=tcp://172.17.0.2:4369
MQ_PORT_15672_TCP=tcp://172.17.0.2:15672
MQ_PORT_4369_TCP=tcp://172.17.0.2:4369
MQ_PORT_4369_TCP_PROTO=tcp
MQ_PORT_5672_TCP_PROTO=tcp
MQ_NAME=/buildapi/mq
MQ_PORT_15672_TCP_PROTO=tcp
_=/usr/bin/env

As you can see the proper host to look at is 172.17.0.2 instead of localhost. Luckily, since these are environment variables, we can just insert them into our configs by name, rather than hard coding them.

After this step, I was still getting a kombu error, which was caused by not having proper login credentials for the amqp. In order to fix this I had to add a userid and password to the config.ini and selfserve-agent.ini files in buildapi. However, buildapi/buildapi/lib/mq.py does not open the kombu connection with the userid and password parameters filed in, so I had to patch this file. I also opened a bug to handle this patch, or to have documentation generated for the proper procedure. The patch is simply:

@@ -21,16 +21,18 @@ import logging
 log = logging.getLogger(__name__)
 
 class ConfigMixin(object):
 
     def setup_config(self, config):
         self.heartbeat = int(config.get('mq.heartbeat_interval', '0'))
         conn = Connection(config['mq.kombu_url'],
                           heartbeat=self.heartbeat,
+                          userid=config['mq.userid'],
+                          password=config['mq.password'],
                           transport_options={'confirm_publish': True})
         self.connection = connections[conn].acquire(block=True)
         self.exchange = Exchange(config['mq.exchange'], type='topic', durable=True)
 
     def get_queue(self, queue_name, routing_key):
         return Queue(queue_name,
                      durable=True,
                      routing_key=routing_key,

Once all of this was fixed and setup, it appears that buildapi and selfserve-agent were able to connect to the amqp perfectly fine!

Left to do on buildapi-app is to:

  • Test that buildapi and selfserve-agent are truly connected and able to exchange over the amqp
  • Setup the databases properly and load them with temporary data
  • Test the entire buildapi application by running similar procedures that should work in my local setup

Updates to this setup can again be found in my user repo http://hg.mozilla.org/users/jozeller_mozilla.com/vagrant-docker-setup/