Yearning for Nosetests

I have begun writing unit tests using the python module unittests. This is not ideal because nosetests is a built in unit testing framework with pylons that does fancy things like making the mocking of internal app attributes much more seamless, or even possible.

Currently I have done the following:

  1. Determined that testing to make sure that /self-serve/{branch}/test_builders successfully calls selfserve.new_build_for_builder is not necessary since this functionality is for Pylons to handle, and isn't really effected by the new functionality needing the unit tests in the first place.
  2. I have been debating what the best way to test that selfserve.new_build_for_builder adds an entry to the mq is. On the one hand, if I just had nosetests working, I could simply mock up the function that grabs from the mq, then I could call the entry function and see that the mq entry is correct. However, if nosetests is not working, I seem to be left with few choices, each with their own problems:

    1. I could mock the function that does the mq entry, and then simply check that the object being passed to carrot.messaging.Publisher.send() has all the correct information necessary for the desired functionality. The problem with this is that mq.py is initialized at the start up of buildapi, and at that time carrot.messaging.Publisher is initialized with various config info. The more I dig into this, the more it appears that this is not a rabbithole I should continue to explore.
    2. I could create a new user to access the mq (through RabbitMQ) and then with buildapi and the mq started up, I could use urllib to send a custom request to buildapi, and then watch the mq for the entry, grab it and then verify it. This approach is seriously not ideal. For one, it requires manual setup from a user before being able to run the test, and so it's not easily portable. And two, it's not isolating just the function we want tested, and leaves the door open to other errors in functions unrelated to our test.
  3. I've consolidated the following two unit tests into a new one, which is simply stated as "selfserve.new_build_for_builder requests an entry that is complete and accurate"

    1. selfserve.new_build_for_builder adds an entry to the mq
    2. selfserve.new_build_for_builder's mq Entry is complete and accurate

So to recap, my revised list of unit tests to complete are:

  1. selfserve.new_build_for_builder requests an entry that is complete and accurate
  2. selfserve-agent.do_new_build_for_builder is called and see's all info from selfserve.new_build_for_builder's mq entry
  3. selfserve-agent.do_new_build_for_builder enters info into database correctly

Things not to test for:

  1. /self-serve/{branch}/test_builders successfully calls selfserve.new_build_for_builder

I just sent an email to catlee to ask if he had any guidance to offer on nosetests, given that 3 years ago he seemed to have success with test_builds.py

 

Making Progress

With my last post I made sure that my development environment was finally in working order and I could begin developing the patch needed for bug 793989. So, I've delved back into the partial patch that catlee has already written, made sense of exactly what is happening with all the pieces, and how they relate to a different sort of buildrequest entry, namely selfserve.rebuild_build. I now have a pretty solid idea of what a complete/accurate set of schedulerdb entries should look like for the new functionality required by this bug. Now I am taking the time to actually develop tests *before* writing the patch. This is a bit of a new thing for me, but I can definitely see the major advantages to testing first.

As far as I can tell, there are 5 main things I need to test for:

  1. /self-serve/{branch}/test_builders successfully calls selfserve.new_build_for_builder
  2. selfserve.new_build_for_builder adds an entry to the mq
  3. selfserve.new_build_for_builder's mq Entry is complete and accurate
  4. mq.do_new_build_for_builder is called and see's all info from selfserve.new_build_for_builder's mq entry
  5. mq.do_new_build_for_builder enters info into database correctly

A few questions are still floating around though. Pylons is setup to run with nosetests, which is really nice because you can load a partial WSGI app and then mess with its internals to test everything as it would be in a real app. However, I have never been able to get this to work successfully. So the question is currently, how long should I spend trying to figure out nosetests? If I decide to forgoe nosetests, I can easily use unittest as I have for other unittests before.

The plan is to contact catlee to ask if he has any additional knowledge pertaining to running test_builds.py with nosetests given that he is one who wrote it 3 years ago. If not, I am going to continue on making the unittests with unittest. Unittest should allow me to do most, if not all, of the testing I need to do, it just needs a little more finessing to get it just right.

I am going to start on the unittest version of tests before hearing back from catlee, since they are easily portable to nosetests and I won't be waiting to get it done.