Whatever Happened to Reuse?

Author: by Steve Adolph

While many companies justify their investment in object technology by citing productivity gains realized from systematic reuse, few have seen productivity increases. Software development economics don't generally justify reuse. Without reuse, why bother with object technology?

Ask most people why they are interested in object technology and more often than not the answer will be reuse. Nearly a decade ago, Electronic Data Systems performed an experiment in which they redeveloped a manufacturing system that was originally written in PL/1. A team of Smalltalk programmers received the same specifications and test suites as the PL/1 team had been given and proceeded to replicate the manufacturing system. The results were impressive. The original PL/1 system consisted of over 265,000 source lines of code (SLOCs), required 152 staff months of effort, and took more than 19 calendar months to develop. The Smalltalk system consisted of 22,000 SLOCs, required 10 staff months of effort, and took only 3.5 calendar months—overall, a 15-fold improvement in productivity, attributed to the systematic reuse of software components (see David Taylor’s Object Oriented Information Systems: Planning and Implementation, John Wiley and Sons, 1992).

Even though these numbers may appear inflated for everyday developers, this was still powerful stuff in the hands of managers desperately looking for a technological solution to cope with their project backlogs. It made sense that developers should construct programs from pre-built components rather than handcrafting programs line by line. The electronics industry, which assembles new products from component catalogs, is often cited as the model the software industry should emulate. Brad Cox coined the term Software (IC) Integrated Circuit to describe this new style of software construction (see Brad Cox’s Object Oriented Programming: An Evolutionary Approach, Addison-Wesley, 1986). Corporations began investing huge sums in retraining and retooling to adopt object technology, often using the economics of reuse to justify the investments.

A decade later, we have reusable frameworks for creating user interfaces, and reusable libraries for network communications, and data storage, but few components for creating applications. In general, applications are still handcrafted. We certainly have not seen an order-of-magnitude increase in productivity, and, in fact, we’ve been lucky to just get a few percentage points of annual productivity improvement. Reasons often given as to why we have not seen widespread systematic reuse in the software industry range from “You’re using the wrong language,” to the “not invented here” syndrome, to the final catch-all excuse, “lack of management support.”

I don’t believe these reasons fully explain why we do not see widespread reuse in software development. Most programmers and managers enthusiastically want to embrace systematic reuse—but in truth, the economics of software development work against its adoption.

What Is Reuse and What Is Reusable?

First of all, what do I mean by “reuse?” If you ask five programmers what reuse is, you’ll get eight different answers. I like Will Tracz’s definition in Confessions of a Used Program Salesman: Institutionalizing Software Reuse (Addison-Wesley, 1995): “Software reuse is software that was designed to be reused.” Note the emphasis on the phrase “designed to be reused.” This distinguishes software reuse from software salvaging, reusing software that was not designed to be reused. Many people regard salvaging as reuse, because they avoid rewriting code by salvaging classes and functions from existing applications. This form of opportunistic reuse is definitely beneficial to the software development organization, but it is not the systematic reuse that industry pundits had in mind.

What most pundits (especially Brad Cox, when he proposed Software ICs) were envisioning was integrating reusable components—pre-built, fully encapsulated software—into our applications the same way that hardware engineers integrate chips into boards. When the order-of-magnitude productivity improvement predictions were made, the predictors were thinking in terms of software components that had been especially designed for reuse.

This was a lovely idea and, with the widespread adoption of object-oriented programming languages, one whose time had come. Yet today, most cases of reuse are actually software salvaging. Despite many who say reuse has not reached its full potential because programmers are opposed to it, I firmly believe most programmers would readily embrace reuse. Thus, the forces opposing reuse are not due to programmers’ fear of change, nor to programmers’ notoriously large egos. For many software development organizations, reuse simply may not make economic sense.

Reuse and Staying in Business

There is a saying that goes “It’s hard to drain the swamp when you’re up to your butt in alligators.” A corollary might be “It’s hard to think long-term when you’re fighting to make next week’s payroll.” A reusable component can cost three to five times more to develop than conventional software because the reusable component cannot make simplifying assumptions about its environment. At the 1998 conference on Object-Oriented Programming, Systems, Languages, and Applications (OOPSLA) in Vancouver, Luke Hohmann offered a wonderful example to illustrate this point. Consider a game program that requires a six-sided die. As an old-time C/C++ programmer, I can write that in one line, call rand(), do modulo arithmetic on the result, and return an integer between 1 and 6. No problem—10 minutes, tops!

Now, what if I want this to be a reusable component? First, I can’t assume the die has only six faces, so I have to design the Dice class to take a parameter that tells it the number of faces. Pretty simple, but a bit more work than my simple one-liner.

What value should each face of the die have? Many games have dice that turn up messages, so I probably want to construct the Dice class to select an entry from a pre-defined list of values. This is starting to get complicated. Then there is the issue of error handling. Because the Dice class cannot make any assumptions about its environment, it will have to use some kind of exception handling mechanism to report problems. Clearly, this is going to take me a lot longer than my simple one-liner to design, code, and test. Finally, I’m going to have to write a programmer guide that explains how to use the Dice class.

The economic justification for spending this extra effort to design the Dice class is that your development organization can amortize the cost of the reusable component over several generations of product or over several different products. This means taking a long-term view and regarding the extra time and effort creating the reusable Dice class as an investment that will generate savings for future systems.

Long-term thinking and vision is often hard to come by in an industry where market windows for products may be open for fewer than six months and the first one to market takes all. This isn’t due to narrow vision and greed on the part of management; it is simply the economic reality. In a small startup, people may have mortgaged their homes to finance the company. They aren’t thinking about how the company will compete in the long term—they’re hoping they will have a product to sell before the money runs out.

A company working to create reusable components for its product will be beaten to the marketplace. It may have a long-term economic advantage over its competitors, but only if it survives for the long term. If the faster moving, handcraft company gets to the marketplace first, it will establish marketshare and may be able to crowd the first organization out.

Few and Far Between

There are, however, counterexamples to this software development model; some organizations are currently achieving very high levels of reuse. An excellent example is the Software Engineering Laboratory (SEL) at NASA Goddard, which routinely achieves an astounding 75% or higher reuse level (see “Evolving the Reuse Process at the Flight Dynamics Division (FDD) Goddard Space Flight Center” by S. Condon et al. The article is available from NASA/Goddard’s web site at http://sel.gsfc.nasa.gov/doc-st/tech-st/sew21/reuse-pr/sew21gss).

This level of reuse is being achieved in the fairly specialized problem domain of spacecraft flight dynamics. The SEL adopted a domain analysis approach, carefully studying and modeling to create the reusable components. The process of domain analysis is similar to requirements analysis; the domain analyst reviews the requirements for past, present, and, when possible, future space missions. The specifications for reusable components are derived from this information. Construction of applications for a specific mission then becomes more configuring components from a reusable asset library than new development. The key aspect of domain analysis is that requirements extend beyond a single project.

The results speak for themselves. Prior to the construction of the reusable asset library, it took 58,000 hours to develop and test an application for a typical flight dynamics division mission. A recent flight dynamics application configured from the reusable asset library took just a tenth of the effort.

Of course, this tremendous improvement in productivity did not come free. Construction of the asset library required 36,000 hours of domain analysis and 40,000 hours to design and implement the components. They started designing the reusable asset library in 1992, and configured the first application from the library in 1995. The SEL estimates that it will have recouped the library development costs by its fourth mission.

NASA has, as well it should, the luxury of being able to take a long-term view. It is hard to imagine a typical software product company having the economic resources to perform a domain analysis. Can you visualize your development organization being able to divert its best developers from current production to create reusable assets, no matter what the long-term benefits are? The initial investment required to develop a reusable library or framework is an economic barrier to the adoption of a systematic reuse program. But hold on a minute! Why am I talking about building your own reusable components? After all, very few hardware vendors make their own chips. Why not just buy your reusable components off the shelf?

Where Is the Component Market?

We were supposed to get that productivity jolt from third parties who would create catalogs of reusable components, which the equivalent of software original equipment manufacturers would then assemble into applications. The sad fact is, not many components can be reused to the extent we’d like.

Every instinct we have as programmers screams out that there must be common components for software applications. Most programmers know that it’s rare to create something truly new. Good software professionals keep a mental backpack filled with previously used concepts they can apply when a familiar-sounding “new” problem comes along. Object technology seems to offer the tools to reuse these concepts as reusable components. Unfortunately, we may have confused the idea of common concepts with reusable components.

As an example, most information systems have a concept of a customer, and most customers have the same behavior: They purchase goods and services from an organization. The behavior of these customer transactions is rigidly specified by law and accounting rules. Is it possible to write a reusable “customer” component that you could sell? While the concept of a customer may be similar, the implementations can be wildly different. Could a financial institution use the same customer component as a petroleum company? Could two companies in the same line of business use the same customer component? Companies have different policies for handling customer transactions, so component vendors must either seek the lowest common denominator in their customer components or let the client modify the component. Most experts agree that if more than 20% of a component must be reworked for its new context, it is more efficient to start from scratch.

There has been some success in creating a commercial component market. Numerous infrastructure components are available for easing the construction of applications. A recent catalogue listed Windows development components such as grid controls, charting, and data visualization. Other vendors offer components for data management, thread management, and data sharing between applications. These types of components have helped developers deal with the increasing complexity of the application environment. However, commercial components have not catalyzed a fundamental change in the industry from handcrafting to assembly-line production—with the accompanying productivity increase.

Where Did All the Money Go?

When it comes to object technology, many companies appear to be riding a policy roller coaster, just following the ups and downs of wherever their software vendors take them. When you ask why they are investing in object technology, they usually answer that they want to improve productivity through reuse. The problem with this reply is that if object technology is justified solely on the basis of reuse, it can only lead to disappointment and frustration when a company’s investment does not achieve huge productivity gains. To date, I have seen several large companies abandon or dramatically scale back their client/server object-oriented strategies and revert to their rusty but trusty mainframes. These Fortune 1000 companies in telecom and financial services justified their failed projects on the promises of tool vendors and MIS managers. When the increased productivity did not materialize, or worse, when productivity actually dropped because the staff was learning the new technology, the projects were abandoned or scaled back. It is not fair to dump the blame for these project failures on object technology, but overstating the benefits of reuse probably did much to distort project risk management.

Systematic reuse for many companies is not economically justifiable because of the high up-front costs. Further, an open market of reusable components may be impeded by the fact that much software is simply not reusable. There are many common concepts, but few common components.

Is object technology, then, beneficial only to companies that can take a long-term view? In a word, no. The object paradigm has encouraged programmers to shift their focus from the implementation of concepts to the relevant abstractions of a concept—in other words, thinking in terms of black-box components, separating the interface of an abstraction from its implementation. This leads to more resilient programs.

Furthermore, while most of the reusable components available in the marketplace are infrastructure components, they do help programmers cope with the ever-increasing complexity of the user interface, data storage, and distribution.

Reuse and object technology are not the technological silver bullet we are forever searching for. If your justification for investing in object technology is based on an order-of-magnitude improvement in productivity from reuse, then save your money, because it’s not going to happen. However, if your justification is based on the premise of progressive improvement in productivity and software quality, then your expectations are more realistic. Finally, features of objects that facilitate systematic reuse also facilitate code salvaging. While salvaging code is not the same as reuse, every line of code you can salvage is a line you don’t have to write. Now that’s worth real money.