Sunday, May 14, 2006

How about making unit testing part of the language? I would like to make unit tests a first class part of a language rather than bolting them on with annotations or encoding their function in class names. I don't have a strong argument against the minimalist language folks who like to keep the core language small, other than I have feeling it would make for a more powerful and useful language in the long run, even if I can't think of compelling reasons why at the moment :-) One feature I would like to add is the ability for one class to say it is the unit test for another class. This would allow: * The unit test class to have internal access to the class under test. * An explicit test to make sure the often volumous test code is excluded from deployment images. * Tools to automatically run tests and report results. * A unit test class to be in a different package than the class under test. So why did this burning issue come up? Because when programmers unit test their C++ code they always wonder how they should do it. Should they make classes friends so they can peek at their privates? Should they make everything public? Should the they be able to call private methods? Should they access member attributes? Should they make two builds, one with #ifdef UNIT_TEST type code and one without out? How do you deal with static methods? There are really an endless number of questions because every situation you encounter in your program needs a way to make it cleanly and easily unit testable. There are answers to all these questions, but the key points for me are: 1. Maintain data hiding. 2. A test must be able to do anything and everything necessary to verify correctness. 3. Code under test should have no idea it is being tested. The engine that pulls the train here is the desire for data hiding: only expose the methods and attributes needed to fulfill the public contract of the class and the public contract should be as small and single purpose as possible. Exposing your privates could get you arrested, but more importantly it creates opportunistic dependencies that will make change much harder down the line. Programmers will use whatever you expose and you will have a hell of a time making changes once code is in wide use in deployed products. I hate it when people say I never do X because once we did X and it bit us bad. So I won't say that. Not being vigilant about data hiding has bit me many many times, but here's one particularly memorable mistake I made. On one large project I left public what I thought was a harmless attribute that nobody would really use. Why make an accessor or class to represent it? Later I wanted to change the attribute to be a method because we wanted to radically change how it was computed and used and we also had a new customer requirement to log accesses to this attribute. No problem I thought. Silly me. I made the change and the build immediately busted. After an analysis of the code base it turned out a lot of programmers had started tweaking this attribute because it made some difference in an important algorithm. Oh oh. This attribute was used in over 7000 places! More importantly the change would impact numerous groups. What this meant is that I could not go in and change it because an impact analysis had to be made before the change could be made. An "impact what?" you are probably asking. You see, in a large project, one with hundreds of people distributed throughout the world, one with millions of dollars of penalty clauses for down time or performance failures, you can't just make a big change without following the proper procedure. Say a bug fix will touch 5000 source files and people freak. All the impacted groups have to take a look at the change and figure out the impact on the code, on reliability, and performance. Full many week regression tests must be run. And of course this must be scheduled. And of course everyone is in a release crunch so they can't get to it until the next release-- next year. So the change was canceled, it was too risky. In a small project you could have made the change, but in larger projects with a lot on the line the world is often very different. So I am generally big on data hiding. I don't make everything public and hope programmers will do the right thing and pay attention to warnings and documentation. I wear a seat belt and I hide my data behind protective methods and classes. I sleep much better that way and I don't have to spend so much time telling a long list of managers what an idiot I am for making problems for them. What this implies is you don't make methods and attributes public just so a test has access to low enough level details that the test can verify the result. For example, if you add an item to an object that contains a list, do you need a corresponding get method to verify that the item was properly added? What if your public contract doesn't require a get method? Should you added it just for the test? Nope. You would be exposing more than you need to which creates dependencies which makes code harder to change. In C++ you have the friend declaration which allows another class to have internal access to a class. This is perfect for test classes. My unit test can reach in and verify that the item was added to the list without exposing unnecessary functionality in the public contract. Perfect. So why am I not happy? 1. Not every language has this feature, Java for example. 2. It's not a first class feature because it doesn't state the relationship explicitly. A friend is not the same a unit test relationship and a unit test may have more or fewer rights and capabilities than a friend. That ability is lost by just calling everything a friend. 3. And a more subtle and some would say silly point is I want to see friendship reserved for production code, not test code that isn't part of the product shipped to customers. Hopefully someone can think of some more reasons why unit tests could/should be integrated as first class citizens of a language. I'd love to hear your ideas. comment[] 5:15:37 PM

© Copyright 2006 todd hoff. Last update: 7/11/2006; 12:33:25 PM.