Wednesday, June 14, 2006

Reanimating Zombie Code - The Art of Source Code Necromancy In sites like sourceforge and in the source code repositories of corporations all over the world "live" millions upon millions of lines of zombie code. Zombie code isn't alive anymore because its original animating soul has been lost. That's crazy you say, source code doesn't have a soul, most people don't even think animals have a soul, and a lot of people don't even think souls exist, so how can source code have soul? Ok, you go me there. What I mean by source code having a soul is: As long as there are people around who understand the source code, change it, improve it, evangelize for it, and faithfully act as intermediaries between the source and users, then the source can be said to be alive. Just having a lot of users doesn't make source alive. We use hundreds of applications (and libraries) that nobody has a clue how they work anymore. Heck, nobody may even have the source code anymore as it was lost in the fog of organizational churn. The application just works and that's all that matters...for a while. That blisteringly fast fortran matrix multiplication package (written by what's-his-name the genius layed off three rounds ago) may be in use for years after it died. That fancy rollup report that talks to the 20 different obscure data sources may be used and admired long after it has died. That website that started with much flourish and seed cash may stay on the web virtually for the end of time, but it is still dead. That an application works is all that matters until something happens and it is officially pronounced dead or just floats unmourned across the river Styx. What forces are afoot that create zombies out of once living code? 1. You might move to a different platform and nobody knows enough about the source to port it. It's just dropped. 2. A user may want a minor change and nobody is around to make it. Then they create their own application and people gravitate to the project under active development because people know when something is alive or dead. 3. Or perhaps there's a serious bug and nobody can figure out how to fix it anymore. Performing an autopsy is a lot easier than repairing. 4. Or maybe a group flush with an urgent mandate and a big budget went looking for an application and they couldn't find it because it had no internal champion. So they built their own...again. How does code become zombified? Mergers, firings, people moving on, new people moving in, technologies moving on, poor project transfer practices, fading memory, no documentation and so on. Some of the most interesting causes of zombification to me are much more subtle: 1. Even code that is part of an active project that practices collective code ownership can become zombified through inattention. 2. It's impossible to transfer a project of any complexity on to other people. A project always has development hot spots. Code that is being worked on has group memory and attention keeping it alive. But as development surges forward many parts of the code, over time, become less visited. And because nobody visits the code anymore it is not passed on between programmers. It becomes zombified. That's why you'll see people even on the same project reinvent the same thing over and over again. Not to go over the edge (I know, too late), but every line of code tells a story. When you are tasked with handing off a project to another group you realize the enormity of the the effort and the absolute impossibility of it. There are thousands of lines of code, perhaps tens of thousands, perhaps millions. Can you really go through them all and explain what you mean? And using good names and all that doesn't substitute for the forces that generated the millions of decisions that went into every corner of the project. A flood of memories will spring up when someone asks you what this subsystem does and why didn't you use another approach instead. To explain what you do you'll respond with a story. That's how humans communicate. Stories. You won't show them a dozen pages of unit tests. You won't show them a wall size UML diagram. You won't show them a hundred page product requirement document. That's not what they want to know and you know it. They want to know directly why this code is the way it is and any implications it may have. They really want a story that will answer any questions they may ever have. You might tell them a story like, "Group X said they were going to get their part done a year ago and they didn't. They also said that it would do this that and the other thing, but didn't. So we ended up making a whole series of changes to make up for it. Not ideal, but we were stuck because this system talks to another system which talks to that system which talks to these two systems a thousand miles away so that means we have to do that stupid thing in our code here." But that's not the real story. You could easily do a 10 minute deep dive on any one of these issues to explain all the forces involved. Code comes down to personalities, politics, budgets, schedules, technologies, quirks, competition--thousands of forces, both hard and soft, playing themselves out on the human stage. You could almost go through this entire process for each line after line in the code. After awhile you just give up and skim the surface. Information is lost at every step so the overall power of the project is reduced proportionally. But isn't this the kind of information you really need to pass a project on? I think so. It's the kind of information I've wanted in the dozens of times I have taken over large code bases and it's the kind of information I would love to impart when transferring a large code base from my care. But it rarely happens. Source code zombification isn't the exception, it is the rule. So how do we keep our source ensouled? Here are a few suggestions of varying amounts of sanity: 1. Don't build it. 2. Buy it. 3. Make a no-brainer install. 4. Make the source repository searchable. 5. Document. Document. Document. 6. Encourage talking. 7. Use a good naming and other development practices. 8. Create a sustaining organization. 9. Collective code ownership. 10. Create a movie. 11. Create a life story. 12. Bring it back to life. Don't build it. If it was never alive it can't become a zombie. Maybe you don't need it after all? Buy it. Transfer the worries of zombification on to another company by buying their software. Make a no-brainer install. Nothing becomes a zombie quicker than something hard to use and install. People will give up on immediately. Make the source repository searchable. By making code searchable it becomes discoverable which means when people need it they have a chance of finding it. Document. Document. Document. Documentation is never enough, but it is like a defibrillator, it gets the heart started but unless the body is healthy enough it won't survive. Encourage talking. When people talk and communicate they keep things alive. Use a good naming and other development practices. This is the same class as documentation. It's not enough, but it helps. Create a sustaining organization. Have an organization whose job it is to maintain and keep software alive. Many important problem solving projects start at the edges of organizations and then go away because there's no home for them when the organization churns. Create a corporate home for software created at the edge. Collective code ownership. An agile practice that does a pretty good job. Create a movie. An old fantasy of mine has people make a movie while coding so they can explain through story telling what they are doing, why they are doing it, when they are doing it. Most people don't like to write, ie document, but they might tell a story (externalize their internal thought process) as they are coding. Developers can watch these movies to learn about the code and make the stories their own. And they of course will make their own movies as they change the code. Create a life story. This is a relatively new fantasy of mine made possible by fast wireless networks and lots of storage. Instead of just creating a movie of the coding process, we create a movie of all the meetings, discussions, and other forces that go into the making of the software. All this tacit context is never documented just is most responsible for the form and content of software. Imagine being able to watch and highly edited life history of software from the POV of all the people involved? Wouldn't that keep source code alive? Now that's a story. Bring it back to life. Bringing dead or zombified code to life is the job of powerful necromancer. It takes an immense amount of work, which is why it is rarely done. You have to reanimate the code in your own mind to bring it back to life. It takes endless hours of studying the code, finding the proper tools machines and environments, running the code, seeing how it's used, and interviewing people involved with the project. It's a good feeling though once you are done. comment[] 2:25:49 PM

© Copyright 2006 todd hoff. Last update: 7/11/2006; 12:32:09 PM.