I am very close to having the buildapi-app docker container working completely. I left off last not having selfserve-agent setup, and having a kombu error.
In order to setup selfserve-agent properly, I had to include a selfserve-agent.ini file in the base of the docker file to be used by selfserve-agent.py when called with: python buildapi/buildapi/scripts/selfserve-agent.py -w; Additionally, I included a simple bash script to ensure that the container is able to launch both processes side by side without blocking one another.
The error I was having with kombu was because I did not have rabbitmq-app running. Kombu is used (as carrot was before) to make a connection to the amqp that rabbitmq sets up as an mq. After getting rabbitmq-app up, it needed to be linked with buildapi-app, and once it was it became clear that localhost was not the proper host for buildapi or selfserve-agent to attempt to find the amqp. When docker links containers, it allocates all the ports and IPs for them. It makes these new connections available to you in the form of environment variables. Once I had the 2 apps up and linked by running:
docker run -d -p 5672:5672 -p 15672:15672 -p 4369:4369 -name rabbitmq rabbitmq-app
docker run -t -i -p 8888:8888 -link rabbitmq:mq -name buildapi buildapi-app /bin/bash # bash so that I can play with the variables
Then I was able to run env and see the environment variables that docker setup:
As you can see the proper host to look at is 172.17.0.2 instead of localhost. Luckily, since these are environment variables, we can just insert them into our configs by name, rather than hard coding them.
After this step, I was still getting a kombu error, which was caused by not having proper login credentials for the amqp. In order to fix this I had to add a userid and password to the config.ini and selfserve-agent.ini files in buildapi. However, buildapi/buildapi/lib/mq.py does not open the kombu connection with the userid and password parameters filed in, so I had to patch this file. I also opened a bug to handle this patch, or to have documentation generated for the proper procedure. The patch is simply:
@@ -21,16 +21,18 @@ import logging
log = logging.getLogger(__name__)
def setup_config(self, config):
self.heartbeat = int(config.get('mq.heartbeat_interval', '0'))
conn = Connection(config['mq.kombu_url'],
self.connection = connections[conn].acquire(block=True)
self.exchange = Exchange(config['mq.exchange'], type='topic', durable=True)
def get_queue(self, queue_name, routing_key):
Once all of this was fixed and setup, it appears that buildapi and selfserve-agent were able to connect to the amqp perfectly fine!
Left to do on buildapi-app is to:
Test that buildapi and selfserve-agent are truly connected and able to exchange over the amqp
Setup the databases properly and load them with temporary data
Test the entire buildapi application by running similar procedures that should work in my local setup
Updates to this setup can again be found in my user repo http://hg.mozilla.org/users/jozeller_mozilla.com/vagrant-docker-setup/