Dec 17

Why Your Junit Should Look More Like Cucumber

Category: Programming

I’ve recently gone back to writing Java and one thing that’s really struck me again is how JUnit (and XUnit frameworks in general) don’t do much to encourage clean tests. Don’t get me wrong, I’m all about unit testing, but a lot of junit tests aren’t very helpful.

For instance, I regularly see tests that look something like this:

@Test public void testSimple(){
  String[] fields = {"b", "t"};
    MultiFieldQueryParser mfqp = new MultiFieldQueryParser(TEST_VERSION_CURRENT, fields, new MockAnalyzer(random));
    Query q = mfqp.parse("one");
    assertEquals("b:one t:one", q.toString());
    q = mfqp.parse("one two");
    assertEquals("(b:one t:one) (b:two t:two)", q.toString());
    q = mfqp.parse("+one +two");
    assertEquals("+(b:one t:one) +(b:two t:two)", q.toString());
    q = mfqp.parse("+one -two -three");
    assertEquals("+(b:one t:one) -(b:two t:two) -(b:three t:three)", q.toString());
    q = mfqp.parse("one^2 two");
    assertEquals("((b:one t:one)^2.0) (b:two t:two)", q.toString());
    q = mfqp.parse("one~ two");
    assertEquals("(b:one~0.5 t:one~0.5) (b:two t:two)", q.toString());
    q = mfqp.parse("one~0.8 two^2");
    assertEquals("(b:one~0.8 t:one~0.8) ((b:two t:two)^2.0)", q.toString());
    q = mfqp.parse("one* two*");
    assertEquals("(b:one* t:one*) (b:two* t:two*)", q.toString());
    q = mfqp.parse("[a TO c] two");
    assertEquals("(b:[a TO c] t:[a TO c]) (b:two t:two)", q.toString());
    q = mfqp.parse("w?ldcard");
    assertEquals("b:w?ldcard t:w?ldcard", q.toString());
    q = mfqp.parse("\"foo bar\"");
    assertEquals("b:\"foo bar\" t:\"foo bar\"", q.toString());
    q = mfqp.parse("\"aa bb cc\" \"dd ee\"");
    assertEquals("(b:\"aa bb cc\" t:\"aa bb cc\") (b:\"dd ee\" t:\"dd ee\")", q.toString());
    q = mfqp.parse("\"foo bar\"~4");
    assertEquals("b:\"foo bar\"~4 t:\"foo bar\"~4", q.toString());

Now this isn’t the worst I’ve seen, just literally the first unit test I looked at in a random project from the apache foundation. In fact in may ways it’s better than a lot of tests, but already there are problems.

First, what exactly does it mean for q = mfqp.parse("\"foo bar\"~4"); to result in assertEquals("b:\"foo bar\"~4 t:\"foo bar\"~4", q.toString());? Second, testSimple? testSimple what? When I refactor something and now get an error that says that q.toString is now "b:\"foo bar\"_4 t:\"foo bar\"_4" what am I going to do with that?

Now let’s look at a cucumber test shall we? First one I found is this:

Scenario: viewing / on a mobile
Given I am not logged in
When I view the home page from an iphone
Then I am redirected to the mobile page

Well this is pretty nice. What am I testing? That I get the mobile version of the web page when I view it from an iphone. This is understandable. If I get an error indicating that I didn’t see the mobile page I know exactly what went wrong and have a good idea of where to look. More interestingly, if I re-factor the code under test to use a completely different web framework I can still use this test simply by binding statements to the new interface! In the previous case even a fairly modest interface redesign would probably require me to throw out much of the unit test and start over.

Now, this is a bit unfair because the cucumber test is written at a higher level than the unit test, but there is actually a lot you can and should apply from cucumber in your own unit testing.

  1. Write your test to the feature the code provides not to the implementation of that feature.
  2. A test should have three concise sections: setup, action, verification. Don’t put more than one of these in a test
  3. Method names, as well as assert failure text should contain clear human-readable language that makes it clear to someone else what you were trying to do and why it went wrong

All of this can be done at the unit test level and all of it can be done in JUnit as follows:

class MyTestClass
  @Test shouldCorrectlyParseASingleWordQuery(){
    givenAMultiFieldParserWithFields("b", "t");
    thenItShouldGiveMe(...) //I don't actually know what that test was checking exactly, but you get the idea.
  //how to configure the parser
  protected void givenAMultiFieldParserWithFields(String ... fields){
  //how to parse the text
  protected void whenIParseTheText(String x)

When you write in this style you get a lot of benefits almost for free

  1. When the code for parsing the text, configuring the parser, etc changes you only have to change one method instead of every test (less brittle unit tests). In fact if you want to replace this class with something else that performs the same tasks you can transition most of the test directly rather than having to start from scratch.
  2. When someone else reads the code they understand what your class is used for rather than essentially seeing black box in->out pairs (i.e. I know toString should return “abc” but what does that mean?)
  3. It actually serves directly as a guide to someone wanting to know how to use your class since they can map semantic action to one or more method calls


The Author

Michael Smit is a software engineer in Seattle, Washington who works for amazon

Comments are off for this post

Comments are closed.