Yearning for Nosetests

I have begun writing unit tests using the python module unittests. This is not ideal because nosetests is a built in unit testing framework with pylons that does fancy things like making the mocking of internal app attributes much more seamless, or even possible.

Currently I have done the following:

  1. Determined that testing to make sure that /self-serve/{branch}/test_builders successfully calls selfserve.new_build_for_builder is not necessary since this functionality is for Pylons to handle, and isn't really effected by the new functionality needing the unit tests in the first place.
  2. I have been debating what the best way to test that selfserve.new_build_for_builder adds an entry to the mq is. On the one hand, if I just had nosetests working, I could simply mock up the function that grabs from the mq, then I could call the entry function and see that the mq entry is correct. However, if nosetests is not working, I seem to be left with few choices, each with their own problems:

    1. I could mock the function that does the mq entry, and then simply check that the object being passed to carrot.messaging.Publisher.send() has all the correct information necessary for the desired functionality. The problem with this is that is initialized at the start up of buildapi, and at that time carrot.messaging.Publisher is initialized with various config info. The more I dig into this, the more it appears that this is not a rabbithole I should continue to explore.
    2. I could create a new user to access the mq (through RabbitMQ) and then with buildapi and the mq started up, I could use urllib to send a custom request to buildapi, and then watch the mq for the entry, grab it and then verify it. This approach is seriously not ideal. For one, it requires manual setup from a user before being able to run the test, and so it's not easily portable. And two, it's not isolating just the function we want tested, and leaves the door open to other errors in functions unrelated to our test.
  3. I've consolidated the following two unit tests into a new one, which is simply stated as "selfserve.new_build_for_builder requests an entry that is complete and accurate"

    1. selfserve.new_build_for_builder adds an entry to the mq
    2. selfserve.new_build_for_builder's mq Entry is complete and accurate

So to recap, my revised list of unit tests to complete are:

  1. selfserve.new_build_for_builder requests an entry that is complete and accurate
  2. selfserve-agent.do_new_build_for_builder is called and see's all info from selfserve.new_build_for_builder's mq entry
  3. selfserve-agent.do_new_build_for_builder enters info into database correctly

Things not to test for:

  1. /self-serve/{branch}/test_builders successfully calls selfserve.new_build_for_builder

I just sent an email to catlee to ask if he had any guidance to offer on nosetests, given that 3 years ago he seemed to have success with

Making Progress

With my last post I made sure that my development environment was finally in working order and I could begin developing the patch needed for bug 793989. So, I've delved back into the partial patch that catlee has already written, made sense of exactly what is happening with all the pieces, and how they relate to a different sort of buildrequest entry, namely selfserve.rebuild_build. I now have a pretty solid idea of what a complete/accurate set of schedulerdb entries should look like for the new functionality required by this bug. Now I am taking the time to actually develop tests *before* writing the patch. This is a bit of a new thing for me, but I can definitely see the major advantages to testing first.

As far as I can tell, there are 5 main things I need to test for:

  1. /self-serve/{branch}/test_builders successfully calls selfserve.new_build_for_builder
  2. selfserve.new_build_for_builder adds an entry to the mq
  3. selfserve.new_build_for_builder's mq Entry is complete and accurate
  4. mq.do_new_build_for_builder is called and see's all info from selfserve.new_build_for_builder's mq entry
  5. mq.do_new_build_for_builder enters info into database correctly

A few questions are still floating around though. Pylons is setup to run with nosetests, which is really nice because you can load a partial WSGI app and then mess with its internals to test everything as it would be in a real app. However, I have never been able to get this to work successfully. So the question is currently, how long should I spend trying to figure out nosetests? If I decide to forgoe nosetests, I can easily use unittest as I have for other unittests before.

The plan is to contact catlee to ask if he has any additional knowledge pertaining to running with nosetests given that he is one who wrote it 3 years ago. If not, I am going to continue on making the unittests with unittest. Unittest should allow me to do most, if not all, of the testing I need to do, it just needs a little more finessing to get it just right.

I am going to start on the unittest version of tests before hearing back from catlee, since they are easily portable to nosetests and I won't be waiting to get it done.

So after the successes with getting RabbitMQ up and running, there was at least one more thing to be solved before buildapi was really truly entering buildrequests into the schedulerdb. Once I had rabbitmq up and running, it seemed as though buildapi was able to submit a new buildrequest, but in reality it became apparent that while buildapi was connected to the mq, there was something missing from the other side of said mq, to grab and execute the db entries called upon by the queue entries. This is a list of the changes that had to be made from the last post till now:

  • Had to start up as a standalone process up next to the buildapi server
  • In order to successfully start up, a new config was necessary. (catlee had me check out the puppet manifests to understand how should be configured)
  • Add "carrot.vhost = /" under "carrot.hostname = localhost" int he config.ini for buildapi
  • Start up buildapi and then start up
  • To start you have to run it with the wait command to allow it to stay in a loop waiting for input: python buildapi/scripts/ -w -v

Once these additions were made to my local buildapi instance, I was able to verify via SequelPro that once I hit "rebuild" on a given build, that it was indeed added to the buildapi-web queue and then grabbed from the queue by and added to the schedulerdb correctly. Finally the todo now is:

  • Update the wiki doc on Setting up a Local Virtualenv for BuildAPI with the new found instructions
  • Clarify the need from the OP on bug 793989
  • Begin writting up unittests to test for proper entry of new buildrequests into the schedulerdb
  • Write up the needed logic to enter a single buildrequest
  • Review the logic
  • Lather, Rinse, Repeat