So recently, dhh tweeted that he doesn’t really get why cucumber and rpsec are so popular and questioned their benefit outside of a niche audience relative to the high cost of writing with them relative to X-Unit style tests. I’ve used ruby test, rspec, and cucumber extensively and I’m finding it hard to see that point of view.
I think the disconnect may be related to what your perspective is on the use of a unit test. While I think most developers would agree that unit tests are for validating and developing code, I think if that is all you see them for it is essentially like saying that good code is code that performs a task correctly. In both cases there are more factors to consider.
First, tests have to be maintained just like any other code. Unmaintainable tests slow down code refactoring rather than supporting it and as such will generally be discarded and the most useless tests are ones that don’t get run. So at the very least you need to apply the same concepts of readability, maintainability, etc to your tests that you (hopefully) bring to the rest of your code.
A second, and to my mind equally important, use of tests is to document what your system is supposed to do. Large projects with many maintainers and users may not have as much of an issue here. If your framework is covered by thousands of blog articles, books, and examples are widely available then feel free to disregard this advice. For the rest of us, getting as much self-documentation out of a code base as possible should be a major goal.
This is really where both cucumber and rspec outshine XUnit style test frameworks. A cucumber test is a human-readable description of feature of your system that a new coder can read without having to understand the underlying code. Even better, because each statement in the description links to code it provides a direct link between feature documentation and code required to configure, execute, and validate that feature. Finally, error messages have the same level of descriptive context making understanding errors that much easier.
The one point I will concede is that cucumber is not one size fits all. If you are validating an API (as opposed to a product interface like a web page or gui) cucumber is less beneficial because it obscures the API itself and replaces it with human language. In these cases rspec is clearly preferred.
In cases where you are testing the system however cucumber has some clear benefits over rspec:
- It forces you to think in terms of how the product will be used rather than how it is implemented.
- It forces you to describe that behavior concisely.
- It documents the behavior of the system clearly in a way even a non-programmer can understand.
- By doing all of this it makes maintaining tests much easier even as product feature implementations change because implementation is clearly abstracted from the test script