Buildbot-app issues resolved, next installing mysql

I was able to resolve the issues I was having previously with buildbot-app thanks to some help from nthomas and aki. The changes that they suggested and some that I discovered that solved things were as follows:

  • apt-get install -y python-openssl
  • Added a .bashrc with:

     

    • source /Buildbot/bin/activate
    • export PYTHONPATH=/Buildbot:/Buildbot/tools/lib/python
  • Switched to the production-0.8 branch of buildbot
  • Fixed issues with the configuration of buildbotcustom and tools being added to the path
  • ln -s /Buildbot/buildbotcustom /Buildbot/lib/python2.7/site-packages/buildbotcustom
  • Removed ["mozilla-1.9.2", "mozilla-beta"] from release_branches in master_config.json
  • Remove PYTHONPATH export from buildbot-configs/Makefile.master

     

    • This was 'export PYTHONPATH=""', so everytime I ran make anything it reset my PYTHONPATH, negating anything I had added to it

After all that, I was able to get buildbot-app running, but wasn't able to verify if it was actually up and ready to be used. Still left to do is breakout mysql into its own docker container and then link up buildbot to the mysql container to be used.

Lingering issue:

When running ./test-masters.sh in /Buildbot/buildbot-configs/, 2 solitary tests on 2 masters fail, but the logs show nothing at all.

(Buildbot)root@177c6fc1687c:/Buildbot/buildbot-configs# ./test-masters.sh
Checking 22 masters…
bm01-tests1-linux32 bm51-tests1-linux64 bm69-tests1-windows bm70-build1 bm75-try1 bm81-build_scheduler bm81-tests_scheduler bm88-tests1-tegra bm89-tests1-panda bm103-tests1-linux bm106-tests1-macosx bm113-tests1-linux64 bm01-tests1-linux32-universal bm51-tests1-linux64-universal bm69-tests1-windows-universal bm70-build1-universal bm75-try1-universal bm88-tests1-tegra-universal bm89-tests1-panda-universal bm103-tests1-linux-universal bm106-tests1-macosx-universal bm113-tests1-linux64-universal
INFO  – creating "bm89-tests1-panda" master
INFO  – created  "bm89-tests1-panda" master, running checkconfig
INFO  – TEST-PASS checkconfig OK for bm89-tests1-panda
INFO  – TEST-SUMMARY: 22 tested, 0 failed
INFO  – creating "bm51-tests1-linux64" master
INFO  – created  "bm51-tests1-linux64" master, running checkconfig
ERROR – TEST-FAIL bm51-tests1-linux64 failed to run checkconfig
INFO  – log for "bm51-tests1-linux64" is "/Buildbot/buildbot-configs/test-output/bm51-tests1-linux64-0yGBNj-checkconfig.log"
INFO  – TEST-SUMMARY: 22 tested, 1 failed
INFO  – FAILED-MASTER bm51-tests1-linux64, log: 'test-output/bm51-tests1-linux64-0yGBNj-checkconfig.log', dir: 'test-output/bm51-tests1-linux64-0yGBNj'
INFO  – creating "bm81-build_scheduler" master
INFO  – created  "bm81-build_scheduler" master, running checkconfig
INFO  – TEST-PASS checkconfig OK for bm81-build_scheduler
INFO  – TEST-SUMMARY: 22 tested, 0 failed
INFO  – creating "bm70-build1-universal" master
INFO  – created  "bm70-build1-universal" master, running checkconfig
INFO  – TEST-PASS checkconfig OK for bm70-build1-universal
INFO  – TEST-SUMMARY: 22 tested, 0 failed
INFO  – creating "bm75-try1" master
INFO  – created  "bm75-try1" master, running checkconfig
INFO  – TEST-PASS checkconfig OK for bm75-try1
INFO  – TEST-SUMMARY: 22 tested, 0 failed
INFO  – creating "bm75-try1-universal" master
INFO  – created  "bm75-try1-universal" master, running checkconfig
INFO  – TEST-PASS checkconfig OK for bm75-try1-universal
INFO  – TEST-SUMMARY: 22 tested, 0 failed
INFO  – creating "bm70-build1" master
INFO  – created  "bm70-build1" master, running checkconfig
INFO  – TEST-PASS checkconfig OK for bm70-build1
INFO  – TEST-SUMMARY: 22 tested, 0 failed
INFO  – creating "bm103-tests1-linux-universal" master
INFO  – created  "bm103-tests1-linux-universal" master, running checkconfig
INFO  – TEST-PASS checkconfig OK for bm103-tests1-linux-universal
INFO  – TEST-SUMMARY: 22 tested, 0 failed
INFO  – creating "bm106-tests1-macosx" master
INFO  – created  "bm106-tests1-macosx" master, running checkconfig
INFO  – TEST-PASS checkconfig OK for bm106-tests1-macosx
INFO  – TEST-SUMMARY: 22 tested, 0 failed
INFO  – creating "bm69-tests1-windows" master
INFO  – created  "bm69-tests1-windows" master, running checkconfig
INFO  – TEST-PASS checkconfig OK for bm69-tests1-windows
INFO  – TEST-SUMMARY: 22 tested, 0 failed
INFO  – creating "bm106-tests1-macosx-universal" master
INFO  – created  "bm106-tests1-macosx-universal" master, running checkconfig
INFO  – TEST-PASS checkconfig OK for bm106-tests1-macosx-universal
INFO  – TEST-SUMMARY: 22 tested, 0 failed
INFO  – creating "bm88-tests1-tegra" master
INFO  – created  "bm88-tests1-tegra" master, running checkconfig
INFO  – TEST-PASS checkconfig OK for bm88-tests1-tegra
INFO  – TEST-SUMMARY: 22 tested, 0 failed
INFO  – creating "bm89-tests1-panda-universal" master
INFO  – created  "bm89-tests1-panda-universal" master, running checkconfig
INFO  – TEST-PASS checkconfig OK for bm89-tests1-panda-universal
INFO  – TEST-SUMMARY: 22 tested, 0 failed
INFO  – creating "bm81-tests_scheduler" master
INFO  – created  "bm81-tests_scheduler" master, running checkconfig
ERROR – TEST-FAIL bm81-tests_scheduler failed to run checkconfig
INFO  – log for "bm81-tests_scheduler" is "/Buildbot/buildbot-configs/test-output/bm81-tests_scheduler-0DxsKt-checkconfig.log"
INFO  – TEST-SUMMARY: 22 tested, 1 failed
INFO  – FAILED-MASTER bm81-tests_scheduler, log: 'test-output/bm81-tests_scheduler-0DxsKt-checkconfig.log', dir: 'test-output/bm81-tests_scheduler-0DxsKt'
INFO  – creating "bm113-tests1-linux64-universal" master
INFO  – created  "bm113-tests1-linux64-universal" master, running checkconfig
INFO  – TEST-PASS checkconfig OK for bm113-tests1-linux64-universal
INFO  – TEST-SUMMARY: 22 tested, 0 failed
INFO  – creating "bm103-tests1-linux" master
INFO  – created  "bm103-tests1-linux" master, running checkconfig
INFO  – TEST-PASS checkconfig OK for bm103-tests1-linux
INFO  – TEST-SUMMARY: 22 tested, 0 failed
INFO  – creating "bm88-tests1-tegra-universal" master
INFO  – created  "bm88-tests1-tegra-universal" master, running checkconfig
INFO  – TEST-PASS checkconfig OK for bm88-tests1-tegra-universal
INFO  – TEST-SUMMARY: 22 tested, 0 failed
INFO  – creating "bm69-tests1-windows-universal" master
INFO  – created  "bm69-tests1-windows-universal" master, running checkconfig
INFO  – TEST-PASS checkconfig OK for bm69-tests1-windows-universal
INFO  – TEST-SUMMARY: 22 tested, 0 failed
INFO  – creating "bm51-tests1-linux64-universal" master
INFO  – created  "bm51-tests1-linux64-universal" master, running checkconfig
INFO  – TEST-PASS checkconfig OK for bm51-tests1-linux64-universal
INFO  – TEST-SUMMARY: 22 tested, 0 failed
INFO  – creating "bm01-tests1-linux32-universal" master
INFO  – created  "bm01-tests1-linux32-universal" master, running checkconfig
INFO  – TEST-PASS checkconfig OK for bm01-tests1-linux32-universal
INFO  – TEST-SUMMARY: 22 tested, 0 failed
INFO  – creating "bm01-tests1-linux32" master
INFO  – created  "bm01-tests1-linux32" master, running checkconfig
INFO  – TEST-PASS checkconfig OK for bm01-tests1-linux32
INFO  – TEST-SUMMARY: 22 tested, 0 failed
INFO  – creating "bm113-tests1-linux64" master
INFO  – created  "bm113-tests1-linux64" master, running checkconfig
INFO  – TEST-PASS checkconfig OK for bm113-tests1-linux64
INFO  – TEST-SUMMARY: 22 tested, 0 failed
*** 2 master tests failed ***
Failed masters:
  bm51-tests1-linux64
  bm81-tests_scheduler

ImportError: No module named buildbotcustom.misc

In my previous post, I walk through the setup that I have so far for buildbot in a docker container and the issues I am having with it. It turns out that it all seems to revolve around an ImportError for a module named buildbotcustom.misc. I added the following to the Dockerfile based on suggestions:

RUN export PYTHONPATH=/Buildbot:/Buildbot/tools/lib/python
RUN ln -s /Buildbot/buildbotcustom /Buildbot/lib/python2.7/site-packages/buildbotcustom

When I run ./test-masters.sh in /Buildbot/buildbot-configs, it fails all 20 masters. When I look at *checkconfig.log in the test-output/ directory, each and every master is failing checkconfig with this error, which makes sense given that this is the same error given when running make checkconfigs in /Buildbot/build-master/. Seems like a simple path issue, but I am unsure what the next step should be here.

The Dockerfile and other files necessary for the container in question here can be found here: http://hg.mozilla.org/users/jozeller_mozilla.com/vagrant-docker-setup/file/b2c0600df541/buildbot-app

To run this container just pull the directory above and run:

docker build -t buildbot-app .; docker run -i -t buildbot-app /bin/bash;

Then the container will launch! Assuming you have docker already setup on your system. Don't forget to launch the virtualenv in /Buildbot/