There’s a discussion going on the NUnit mailing list about where to put unit testing code. This important question needs to be answered by any project team before embarking on unit testing bandwagon in the context of medium-to-large scale project. Here’s why:
Developing unit tests will end up taking a significant amount of team’s time and will result in significant amount of code. As with any other development and coding guidelines you want to have clear guidelines with respect to organization of unit testing to avoid chaos and confusion. You will want to make it easy for every developer to introduce a unit test – not more difficult than introducing any other piece of code. Finally, you will want to be able to strip unit testing code when you prepare a distribution package of your product. You will also want to exclude NUnit binaries from your distribution package.
My team has spent some time pondering the following alternatives:
Put all the tests in a separate assembly. The idea here is to have a single assembly that would depend on NUnit binaries. When shipping the product this test assembly would be left out of the distribution package. That’s easy. The drawbacks of such solution are that you’ll end up creating a big assembly with lots of tests. This assembly would end up depending on lots of ‘production’ assemblies. The source control of this central test assembly would be problematic since many developers would often need to make concurrent changes to it. Unit tests would have to be limited to public interfaces of production types. We have dismissed this idea because of these drawbacks.
Create a satellite test assembly per each ‘production’ assembly. In this solution it is equally easy to create nunit-less distribution package. It becomes easier for developers to add new tests and not step on each other those in source control. The trouble is, would still be impossible to test internals of production types. This solution would also make the project have twice the amount of assemblies you’d normally need. Source tree and build management issues usually grow with the number of assemblies you have to manage. We have abandoned this idea too.
We’ve ended up the following: Put tests together with the production code, but make them separate. The idea here is to put all unit test code in the same assembly along with the production code, but be able to separate the unit-testing code when building the distribution package. We put all the unit-testing code in the separate namespace just below the namespace of the production code. A namespace MyCorp.MyProduct.MyComponent will have its tests in MyCorp.MyProduct.MyComponent.Test namespace. Furthermore, all the unit-test source files are located in the separate test directory (Test) under the directory that holds the production code. The same Test directory usually holds data files needed to run the tests. This organization solves all the above issues, but makes the generation of test-less distribution package fairly easy as well and makes XML based build files (like NAnt or VS.NET) really shine. Since the organization of test files follows strict rules, nant build files can be changed using XSLT to exclude ./Test/*.cs files from the build. The dependency can also be excluded by the same transform. This solution represent a little bit more work for build-master, but saves a lot of hassle for developers. (I’m not posting the XSLT transform because we don’t have it yet. But stay tuned :)
