Chapter 6 - Testing

Part 1 of this series can be seen here.
Part 2 of this series can be seen here.
Part 3 of this series can be seen here.
Part 4 of this series can be seen here.
Part 5 of this series can be seen here.
Part 6 of this series can be seen here.

There are many plugins available to the Grails developer. Among them are: Easyb Testing, though new to it, a personal favorite of mine (the plugin does not appear to be ready, yet, there is little info on the plugin page. You can still use it. See the official easyb site.), Build Test Data, Selenium, Test Code Coverage, Canoo WebTest, CodeNarc and many others. While it would have been nice to have a more comprehensive chapter on testing, giving those listed above their proper due would get lengthy, perhaps worthy of a book of their own. The author instead focuses on unit testing, including many of the built-in mocking features, integration testing and functional testing using Functional Testing.

The chapter starts off with a bit of history behind automated unit testing, its relationship to Extreme Programming (XP) and Test Driven Development (TDD). He explains how the original unit testing framework is JUnit and that Groovy comes with this, extending and enhancing it, and that the JUnit framework on which Groovy and Grails builds uses the conventions prior to JUnit 4 (meaning before annotations were available). I won't go into detail here. However, I do want to point out that he remarks on one of the best practices of using descriptive test names. Here is a place where I feel easyb, a Behavior Driven Development (BDD) framework, is far superior.

I have another quibble, as well, in this section. He speaks of how many developers write test cases that do not execute against their application code, implying, I believe, that not only do you have to write tests that exercise the production code, but also that the tests must actually be meaningful. I will return to this later. My problem with this section, though, is that he gives an example of a bad test and does not demonstrate what a good test should look like. Someone likely to write a bad test, in my opinion, may not really understand what is wrong with the test as written and not intuit the proper way to write it.

And, this brings up yet another quibble. Here, again, Jon seems to have chosen to roll his own rather than using the automated features included with Grails. Maybe the way test cases were generated changed during the writing of the book explaining the differences between the code he has here in this and other examples. That could explain it, but I think that this should have been revisited and updated with the 1.1 Grails release. I'd like to hear Jon comment on this. The changes are not dramatic, though. The name of the generated unit test for class User, for example, would read UserTests rather than UserUnitTests. Similarly, he extends GroovyTestCase, at least in the example on page 116 and others, rather than GrailsUnitTestCase. In other examples, he does extend GrailsUnitTestCase. In the example testing the MessageController, the framework generates a class that extends ControllerUnitTestCase and he uses GrailsUnitTestCase. The auto-genned classes, too, come with stubs for the setUp and tearDown methods, something not included in his examples.

Jon does list the JUnit assert methods and the additional assert methods provided by the GroovyTestCase, from which GrailsUnitTestCase inherits. These additional assert methods are: assertArrayEquals, AssertContains, assertLength, assertScript, and assertToString. Also included are the convenience methods shouldFail and shouldFailWithCause. These shouldFail methods are then explained further with an example. These are a bit tricky. They take a closure and the closure is expected to throw an exception. If one is not thrown, the test fails. By using the WithCause variation, you can specify the exact type of exception you do expect. Any other exception or no exception at all will cause the test to fail.

On page 121, the author tells us of the convention of where tests are expected to live, and explains that the tests are automatically generated when you use one of the create commands. He then goes on to write his own variation, as stated before, and inexplicably in my opinion. The example here is missing the import for GrailsUnitTestCase. I am not sure that the test was tested!

We are then instructed on how to run the test case and the reporting explained. Nothing here should surprise the professional developer, especially one that has used one of the xUnit frameworks before.

Next, Jon goes on to tell us of the problems we'll quickly encounter with testing a Grails app. Unit tests do not run within the Grails container. This means that dynamic methods and properties are not available to the unit test. This leads to a couple solutions, neither of which is ideal, mocking and running the tests as integration tests. Luckily, as Jon explains, mocking support is built-in through the GrailsUnitTestCase class. He provides a table showing the various methods supporting mocking and a brief description regarding their use: mockController, mockDomain, mockForConstraintsTests, mockFor, mockLogging, mockTagLib and registerMetaClass. Of these, he demonstrates the use of mockController, mockDomain, mockForConstraintsTests, and mockFor. He goes in depth on the latter to explain how the delegate property is used on the closure.

When discussing mockForConstraintsTests the author seemingly violates his earlier assertion that you should write meaningful tests that I mentioned above. I find this rather ironic. Here he tests a constraint (page 127), but to validate only writes:

assertTrue(message.hasErrors())

Well, this means that no matter what error is thrown, the test will pass. Wholly inadequate, in my opinion. It would be better, in my opinion, to take greater advantage of the error property. For example, the test in question could be written as:

assertTrue message.errors.allErrors[0].toString().contains("Field error in object 'app.Message' on field 'title': rejected value []")
assertEquals 1, message.errors.errorCount
assertEquals "title", message.errors.fieldError.field
assertEquals "", message.errors.fieldError.rejectedValue

The first line above could be used instead of the latter three, though takes a bit more work to figure out. You may need to run a test outputting the actual error string. Certainly, your test could get more sophisticated than this and there are other alternatives. All three tests in this example are similarly challenged and the solutions for testing reliably will vary.

After explaining mocking, we are told of the limitations of unit testing, and Jon goes on to discuss integration testing. This is all fairly familiar territory so will not be discussed in depth. He does introduce us to grails.util.Environment in this section, something he promises to delve into deeper in a later chapter. I don't really agree with the reason he is using it, however. He is manipulating the creation of data in our BootStrap.groovy for testing reasons. It is my opinion that the test case should neither depend on the existence or non-existence of such data. Test cases should manage all the variables that impact proper execution.

Finally, we get to functional testing and the Functional Testing plug-in. This is all so simple to install, create and implement, that it almost seems trivial. Just what we developers need! He does stress the trade-off necessary between the level of code coverage and the time it takes functional tests to run. The functional tests are run within the Grails container.

While totally gratuitous on my part, a short discussion of the differences between Webtest and Functional Testing plugins is probably relevant. Webtest had been the original "go to" plugin for functional testing. The Functional Testing plugin seems to be the new favorite. Since many will undoubtedly work on legacy applications that use the former, understanding the differences could be necessary. There is a page on the Grails site that does discuss this, but is rather incomplete. Oh, well.

In my previous post, I complained of having errors where the messages were meaningless to me, even pointing in the wrong direction entirely. I've run into this more in my domain persistence tests than anywhere else. Like a good little Test Driven Developer, I generate my domain, then start updating the integration test before adding attributes or anything else to the domain. My persistence test will look something like:


Somewhere in the test/code loop, I will suddenly see an error that states:


A failure I can understand, but an error, no. Seeing this frequently enough, I realize that my object probably was not saved. The reason for not saving is not obvious. The stacktrace provides no answers. Well, after RTFMing and googling a bit, I came up with a better way to write the persistence test that will tell me when the object is not saved and the reason for it:


The if statement against the save of the first object will cause sysout to be written that can be viewed from test results html of the failing case via the System.out link at the bottom of the page. The above test was written to fail for this example. Viewing the link shows:


This probably should be written to use the fail method rather than a write to sysout. I'll leave that for the future.