Package Properly: Where Your Tests Live Is Important

About nine months ago, our team decided to start using Behavior Driven Design. Rather than making big changes in the build system to bring in a BDD framework, I put together a BDD reporting framework called EasySpec that allowed us to continue to leverage JUnit. One of our goals in trying BDD was to really push the limits of natural language in testing and determine what the limits were for BDD. We ran into limits with our mocking framework, but Mockito solved those problems for us (more on that in a future post).

One thing that we found was that BDD works pretty well for all levels of testing. Many of the BDD practitioners will only use BDD for higher level integration testing. However, I've found that I really like it for unit-level testing as well. Language is important regardless of where. It is certainly nice to understand system behavior in the large, but as a developer I need to understand behavior in the small as well.

We package our unit-level tests in a test source tree that parallels the production code source tree. So, if I am spec'ing out the com.company.foo.NewFoo, then the production source lives in myProject/src/com/company/foo/NewFoo.java and the first spec will end up in myProject/test/com/company/foo/NewFoo_when_X_Test.java.

One unintended advantage of staying with JUnit and EasySpec was that our new BDD tests landed right next to the old JUnit tests. The subconscious communication that this packaging created was wonderful. It removes the step of questioning where the next test or spec should live. The next spec goes into the parallel package. This also communicates that we are going to be driving the same requirements with BDD that we drove previously using TDD methods. In other words, we aren't just writing larger integration tests with BDD. We are writing as much as possible using BDD because again, langauge is paramount to understanding the system.

How do you package your specs? And, have you pushed the limits to find out how low-level you can drive BDD concepts into your design? I love designing in the small with BDD. My production code is better, and the specification artifacts are wonderful. When supporting code writting six months ago, I find it much easier to understand the system if we have EasySpec specifications rather than JUnit tests -- regardless of the level of abstraction under test.

Publishing Build Artifacts With Hudson

Tonight, I found yet another reason to love the Hudson continuous integration server. Publishing build artifacts is way easy. What's better, the latest artifacts are available under a static link. So, it's easy to set a bookmark and always have the latest output available without digging through the server.

I've setup publicly visible builds for EasySpec along with the Groovy example and Java example projects. All of these projects use EasySpec for Behavior Driven Design. A major component of BDD is having the latest behavior report available. So, I have included a build target named "report" in each of these projects. This target simply runs EasySpec to generate the behavior report into a known location in the workspace.

Let's take the Groovy example and walk through it. After every checkin, the Hudson build does

gant clean test report

This cleans the working copy, then compiles everything, runs the tests, and finally generated the EasySpec report. Relative to the working copy base, the report ends up in build_output/reports/EasySpec/index.html

To publish this report, only a couple of simple steps are required. First, go to the project configuration page for the build that you wish to publish artifacts from. In this case, that's Hudson/GroovyExample/Configure. Then, find the checkbox, "Post-build Actions / Archive the artifacts" Enter the relative path of the artifacts that you wish to publish. In this case, that's "
build_output/reports/EasySpec/index.html" Click "Save" and it's all done.

Now, everytime this project builds successfully, the latest EasySpec report is published to a constant URL. If you want, checkout the latest example EasySpec report.

I love Hudson