|
Home | Return to Collected Papers and Articles
Rewrite Or Update?
There comes a time in the lifecycle of all evolving software projects when a decision must be made to keep the existing code-base, or to create something new. There have been many different outlooks in this area, which range from the belief that the first implementation of a product should be discarded as a prototype, to the assertion that re-writing is starting from the beginning and throwing away the knowledge gained. As is the case in most philosophical disputes, there is validity to both viewpoints, but accepting either as a whole truth requires ignoring flaws in the position. In the "throw-away-code" scenario there is an obvious extreme expense in wasted resources. The "throw-away-nothing" concept implies that an original design foundation will always be sufficient for future versions. Software companies have failed by religiously following either philosophy for their flagship products.
One of the incorrect assumptions often made is that there are only two choices in these situations: re-writing the application or continuing along the development path. The truth is that there are several choices, each of which might be applicable to sub-potions of the whole project. The misunderstanding in this regard leads management to make a single yes or no choice, upon which the welfare of the company depends.
A software package can in fact be: updated, migrated, translated, or re-written. Furthermore, a complex application can often be broken into multiple sections which can each be evaluated for the most appropriate process to be applied. The definitions for each of these approaches as follows:
1. Update - To enhance or improve the existing code-base.
2. Migrate - To move the majority of the code-base to a new platform.
3. Translate - To transform the code-base to conform to a new medium.
4. Re-write - To recreate the code-base.
To understand the implications of each of these choices there must be comprehension of the conceptual definition of the components of a complex application. Each of these conceptual levels of abstraction have unique characteristics that impact the efficacy of a given approach. These levels are as follows:
1. Medium - The language in which the application is developed.
2. Platform - The operating environment of the application.
3. System Objects - Abstracted constructs representing the platform.
4. Application Objects - Abstracted constructs representing application domain behaviors and relationships.
5. UI Objects - Abstracted constructs representing user interaction domain behaviors and relationships.
Each of these layers of conceptual abstraction can influence the behavior or constraint of other layers. For example, the UI needs of the application might require a move to a different platform, which might necessitate translation to a different medium due to availability of developer tools. The .NET migration is an example of this.
Within the context of this background, let's examine the assertions made by Joel Spolsky regarding the re-write decision. First off, Joel makes the pretty clear assertion that programmers always want to re-write code to make it more attractive, or because they are confused by what someone else created. I would concur that this is often the case, but it should not necessarily be dismissed as an emotional reaction. Some code IS poorly implemented, or is more convoluted than is necessary to provide identical functionality. Decisions as to such necessity should be evaluated on a case basis. The urge to re-write based upon purely stylistic or personality issues should be overridden. Code clarification however is a benefit to the business, as that maintenance and enhancement of applications represents 85% of development expense.
Regarding development expense, much of Joel's argument is posited upon the belief that "code artifacts" represent application knowledge. He is basically asserting that all of the rough edges and special cases represent the correct implementation of the application. He represents applications as being in essence a rough-hewn "expert-system" that embodies knowledge of the business rules therein. There is definitely a valid point in recognizing that modifications typically reflect enhancement of the "correctness" of the application, be they additional features or bug fixes. In summation, his major assertion is that a re-write would lose all of those "correctness enhancements" and application expertise that they represent, making it necessary to spend the months and/or years of refinement once again.
This represents very good advice as to the importance of understanding the complete behavioral space of a section of code before re-writing it. However, his reasoning is specious when applied as a fixed rule to developmental decisions. One of the areas he vastly understates is the need to move applications to new platforms that are structurally divergent enough to invalidate the original design. The major flaw in his position as I see it is the assertion that it is better to maintain an incorrect design than it is to re-implement with a correct one.
One of the critical aspects to the successful implementation of a procedure (code section) is full comprehension of the solution space. That is a complicated way of saying that you need to know exactly what the procedure SHOULD do. Implementing code without that understanding leads to special casing, patches and tweaks. Or "code artifacts" as Joel calls them. A procedure written with full understanding can define the same or equivalent solution space without confusing artifice, and is equally as "correct". Enhancing and maintaining artifice-free source code is many times more efficient, and therefore represents a massive long-term savings to the developer.
This begs the question: Why do procedures get written without full understanding of the solution space? Most often with business applications this is because the functionality of the application is not fully defined at the time of the implementation. These code artifacts that Joel represents as "nuggets of wisdom" are therefore really desperate attempts to adjust an incorrect procedure to fill in gaps in the solution space. Another cause of incomplete solution space coverage can be poor design, or a lack of design, or an attempt to implement without understanding.
I believe that there are two much more significant causal factors in the historical failures of companies to re-write their major applications, which I refer to as "Feature Glut" and "Kitchen-Sink-ism". My definition for these two factors as follows:
1. "Feature-Glut" - The attempt to add every feature ever considered into the new application's specification. Typically a front-end problem involving sales and product management attempting to create the "ultimate" product.
2. "Kitchen-Sink-ism" - The attempt by the developer to create generalized and flexible abstractions to handle even future functionality. Typically a design problem wherein the engineer attempts to design the "ultimate" framework.
As you can see, the above two challenges are created with the same motivation, the desire to create something truly great, rather than to merely recreate the functionality with a better foundation. I therefore believe that Joel is not correct in blaming failure in these cases upon losing the experience gained, but believe rather that the failure is often times caused by the individuals experience leading to unreasonable project expectations (and hubris). Rather like someone who successfully built a bicycle deciding they will build a race car based upon what they learned. The results are probably inevitable.
Most endeavors' success or failure is based upon how the problem is approached. In the case of re-writing complex applications this is equally true. I think that success is predicated upon several requirements:
1. Clear, fixed, reasonable feature specification.
In some endeavors it is good to "shoot for the moon" because when you fall short you have still attained something. Applied to software specification however, failure is comprehensive, ending with a failed project and lost resources. Thus is the result of building software monuments. The objective must be reasonably attainable (including the time-line!), and must not change during the course of the implementation. I can't emphasize that last part enough. Software is built in layers, like a building. You would not decide to add a second level to a building 50% complete and expect there to be no reworking of what had been done. The same is true in software. Changes and distractions during the process increase the probability of bugs and design flaws. A large project must be engineered from the ground up, and must be re-engineered if the design is changed.
2. Clear, reasonable design limitations.
The designer(s) and engineer(s) must have a clear and reasonable set of internal functionality specifications. Meaning that they must not "shoot for the moon" either, or the results will be the same as with feature-glut. Thus is the result of building software monuments. There is a balance to be found between keeping extensibility in mind, and trying to code the ultimate abstracted framework. Using the construction analogy, imagine if one attempted to build the "ultimate" foundation, whereupon any building could be built, no matter how massive. It would take many more resources to build a house upon it than it would to build upon a simple house foundation.
3. Full understanding of solution space.
As addressed previously, code must be patched and tweaked over time as a result of failure to fully understand the functionality required. These changes are needed because the initial implementation did not fill the solution space. Should the feature specifications and design limitations meet the criteria listed in the above two sections this will only occur due to a lack of understanding on the developer's part. This will place a burden upon the implementer to pursue whatever design and specification resources are needed to provide a complete understanding prior to writing code. For the developer to have a reasonable chance of success in this, the following three sections are also critically important.
4. Manageable application objects. The application level abstractions should be designed in such fashion that they can be intuitively used by the developer. Overly esoteric or complex application level objects lead to developers spending more time fighting with the object than they do implementing the code section. This is equally important in the maintenance and enhancement phase of an application's lifecycle, as that the resource needed for both is proportionate to the intuitiveness of the object. The objects should be easy for the developer to use in fulfilling the solution space of the given procedure. I can't emphasize this enough. There is a direct relationship between intuitiveness of a developer's tools and the level of productivity and number of bugs created. Failures in this area are typically a result of failing to meet the criteria in sections one through three. It should be noted that failure to follow the guidelines in one or more of the sections four through six can directly impact the ability to adhere to the others.
5. Manageable system objects. The objects should be easy for the developer to use in fulfilling the solution space of the given procedure. I can't emphasize this enough. There is a direct relationship between intuitiveness of a developers tools and the level of productivity and number of bugs created. This section is virtually identical to section four above.
6. Manageable UI objects. The objects should be easy for the developer to use in fulfilling the solution space of the given procedure. I can't emphasize this enough. There is a direct relationship between intuitiveness of a developers tools and the level of productivity and number of bugs created. This section is virtually identical to section four above.
In summation, most failures to re-implement complex software systems result as a violation of one or more of the above six criteria for success. To assert that failure is inevitable in writing a complex application is obviously incorrect, else there would be no applications to re-write. To assert that re-writing an application is easy by virtue of having done it once before is equally erroneous. Both positions are examples of over-stating or under-stating the scope of the project. Success is quite possible, as is indicated by the large volume of re-written applications on the market, as well as the developers that move to other companies and are involved in creating similar products.
Home | Return to Collected Papers and Articles
Contact info@artificialingenuity.com
Copyright © 2005 Artificial Ingenuity, LLC
Last modified: June 11, 2005
Initial design by Webinizer, LLC