Algorithme
Services en informatique
Tilff, Liège, Wallonie, Belgique

White Paper: Dynamic Object Oriented Programming

Enabling Applications that Adapt to Changing Needs

Copyright © 1997, Franz Inc. All rights reserved.

Today's software development tools are increasingly unable to keep up with the complex and changing requirements modern applications must meet. Faced with the need to create complex applications faster, software developers are turning to Dynamic Object-Oriented Programming languages for managing complexity and adapting to change. Dynamic OOP languages, such as ANSI CLOS, change the way software is developed. This results in applications that can be rapidly deployed; easily and cost-effectively tailored to the needs of the end user; and change after deployment to meet new user requirements.

Software developers at Price Waterhouse chose a Dynamic OOP language to implement a major overhaul of the company's audit planning process. The Dynamic OOP environment provided by CLOS enabled these developers to quickly standardize and automate the company's audit planning process, increasing the efficiency of Price Waterhouse auditors while improving the accuracy of their audits.

With CLOS, Price Waterhouse was able to create a sophisticated knowledge-based system in one year instead of three. First, the programmers rapidly developed an early prototype of the automation system, called Planet, even before final specifications had been determined. Then, CLOS's ability to easily make incremental changes allowed the system to quickly evolve to its final design through an interactive development process with in-house audit experts and users. "Using CLOS, we were able to quickly change the underlying structure of our program to reflect input from our knowledge experts, and reduced the time required to deploy the application," said Jeff Delisio, Research Scientist at Price Waterhouse's Technology Centre in Menlo Park, California. Planet, running on notebook computers, is helping Price Waterhouse dramatically reduce the time required to plan an audit - from several weeks to as little as one day.

CLOS is also becoming increasingly effective in developing intelligent agents to store, retrieve, analyze, filter and present the mushrooming volume of information becoming available on the Internet. High-productivity, rapid-prototyping and deployment programming tools like CLOS enable these agents to be created quickly and efficiently.

A new Internet-based electronic publication system being developed at MIT's Artificial Intelligence Laboratory under the direction of John Mallery is already being used to make it easier to keep up with the latest information from the White House. The White House Electronic Publication Service applies Dynamic OOP to a networked environment to provide access to a persistent object database via form-processing email servers and the World Wide Web. By changing its behavior on-the-fly, the system enables users to receive only those White House documents of interest to them by selecting a set of preferred categories either through email or custom World Wide Web pages.

According to Mallery, CLOS is an ideal programming environment for the sophisticated National Information Infrastructure applications now being developed precisely because the area is moving so rapidly. The rapid prototyping capabilities of CLOS, including its ability to debug network applications in real time while they are running, enabled a number of dramatic programming feats. The original Communications Linker system, developed for the 1992 presidential campaign, comprised 10,000 lines of code and was written in just 10 days. The first web server interface required just 2700 lines of code and was written from scratch and debugged in about five days by one person. Similarly, the Vice-President's Open Meeting on the National Performance Review was built on top of the Communications Linker system in the four-week period between the announcement the White House web site and the start of the open meeting. CLOS regularly provides a 400 to 500 percent productivity improvement over C++ in real-world development environments, and Mallery's group generated as much as 1000 lines of code a day, enabling them not only to keep up with the rapid pace of change on the Internet, but to stay on the cutting edge of Internet technology.

The Software Development Dance

Software developers like those at Price Waterhouse and MIT's AI Lab are finding they need to change and redesign their software constantly to take advantage of the latest capabilities, as well as stay ahead of the competition. But the tools they are using are quickly becoming inadequate to keep up with the pace of change. Caught up in a never ending dance to provide friendlier, more powerful applications faster, software developers need tools that can better adapt to changing needs.

In 1991, according to an IDC survey, application life cycles averaged 9.5 years, but by 1994 the average life cycle dropped to just 7.1 years, and this trend is expected to continue. Customers are constantly demanding more sophisticated functionality - graphical user interfaces, programs that adapt themselves to the way a user works, more flexible connectivity options, portability and a variety of multimedia capabilities - and the pressure to create them ever more quickly is increasing. In the midst of this, software developers have had to adapt to new technologies and standards for everything from video and telephony to ATM and ISDN to Windows 95 and the Internet. Not only are software developers having to deal with new technologies and new standards, but even the established standards are changing.

Moore's Law

The beat that software developers are dancing to is generated by the continually evolving and accelerating increase in hardware capabilities, a process described by Moore's Law. An empirical law, Moore's Law states that for the same amount of money, computer power will double roughly every eighteen months. CPUs get twice as fast, memory gets twice as large, hard disks get twice as large, networks get twice as fast. All for the same cost. Every eighteen months. And the beat goes on.

The process characterized by Moore's Law has been at work setting the pace of change for some time, and has had a tremendous effect on the software people use every day. The graphical user interface, for example, could not enter mainstream computing until the average PC had the CPU power, memory size, hard disk drive capacity and graphics capabilities demanded by this type of software. Today, PCs with Pentium processors, 8 or 16 megabytes of memory, gigabyte hard drives and sophisticated graphics accelerators are commonplace, and their power completely masks the overhead imposed by the demands of the graphical user interface. Other examples of Moore's Law-enabled software advancements include on-line help systems and documentation, multimedia, Object Linking and Embedding (OLE) and multitasking operating systems.

Managing Complexity

As end-user software has grown in capability and ease of use, software developers have had to pay a price in software complexity and development time. Microsoft Word, for example, has grown steadily from an installed size of roughly 1 megabyte in 1990 to nearly 13 megabytes today. Likewise, AutoCAD has grown from less than 0.5 megabyte in 1989 to about 40 megabytes today. Even Microsoft's C Compiler has grown a hundred times in size from 0.5 megabytes in 1990 to nearly 50 megabytes today. But as software gets more complex, it takes longer to design, and it becomes more difficult to change during development and after deployment.

In light of the evolving user needs, hardware, operating systems, GUIs, standards and technologies, software developers must have the appropriate tools to hit at an ever more quickly moving target. Two years ago, for example, nobody building a word processor anticipated needing HTML-format output. In a rapidly changing, dynamic environment like this, managing complexity requires high-level dynamic development tools that facilitate change.

Static Object-Oriented Programming

Object-oriented programming has become the undisputed successor to the structured methodologies of the 1970s and 80s because of its ability to better manage complexity. Modular software design, code reusability and rapid application development are just some of the many advantages of object-oriented technology.

The premier static OOP language used today is C++. Compared to C, software developers using C++ have reported productivity gains of up to 200 percent. But C++ is hard to use, and requires an inordinate investment in time to avoid memory leaks (that result in poor performance and random crashes) and to tackle exception handling problems.

As applications get larger, C++ tends not to scale well because changes increase the development cost in proportion to the size of the application rather than the degree of change. C++ requires any redesign in the classes and methods to be completely consistent and correct across the entire application source code before a design modification can be compiled, let alone tested. Any change in the classes causes the entire source to be recompiled, which in a large system can take hours. For this reason, programs written in C++ are monolithic and immutable, making it hard to redesign an application during development, and even harder to maintain and reconfigure software systems after an application is delivered.

This "brittle class" problem (so called because trying to change any element seems to make the whole application break) remains a long-term weakness of C++. It makes it hard to incrementally develop software, impacting productivity, usefulness and maintainability of the final product.

In C++, objects are both a blessing and a curse. While a C++ program allocates objects all the time, it is not always clear when those objects go away. Very often objects refer to other objects in very complicated ways. Yet C++ does not provide any form of garbage collection to reclaim the memory used by defunct objects. As a result, instead of working on the problem the application is designed to solve, a lot of a software developer's effort - about half the total time needed to create an application - goes into dealing with a low-level systems problem that does not exist when using a dynamic language. More often than not, C++ developers resort to changing the application design to overcome memory management difficulties, resulting in an application that's less useful and less understandable.

Dynamic Object-Oriented Programming

Dynamic Object-Oriented programming is a software development technology that enables applications to be tailored during development and after deployment without access to source code. Made practical by the continuing hardware evolution predicted by Moore's Law, Dynamic OOP languages are much more effective than static OOP languages for managing complexity and adapting to changing needs.

With Dynamic OOP languages, the amount of work necessary to make a change is proportional to the degree of change, not the size of the application. New objects, new classes and new behavior can be added on the fly, and unlike static OOP languages, Dynamic OOP applications do not have to be rewritten to accommodate any change.

Dynamic OOP is the enabling technology for user-evolved software. Developers can incrementally test working prototypes with users, as Price Waterhouse did, resulting in significantly more usable applications. Developers have the freedom to try new things even when they aren't completely consistent with the rest of the application. Inconsistencies may eventually generate errors, but developers can still experiment with running the modified application. With Dynamic OOP, an application's behavior can continue to evolve after deployment, as the MIT AI Lab's Internet-based Communications Linker does, and intelligent agents can reconfigure applications by applying new code while the application is running.

Dynamic OOP uses an additional level of abstraction called meta-data to achieve the magic of incremental change. At run time, an application's design is represented by meta-data that describes the class hierarchy, objects and methods. Changes made to class definitions at run time, such as adding or deleting data members, cause the meta-data, not the objects themselves, to be updated. Whenever an object is accessed, it is first checked to ensure the meta-data describing it is current. If not, the object updates itself to conform to its new definition as described by the updated meta-data, and then normal processing resumes.

The change is incremental, and the application "pays" for the update only once per object access. This ensures the highest performance while retaining scalability. Moreover, this lazy evaluation technique ensures that even applications with millions of objects get their objects updated only on an as needed basis. In this way, the benefits of incremental change can be retained even in very large applications. Building similarly modifiable applications in static OOP languages would be extraordinarily difficult.

Because incremental change requires the system to completely understand the meta-data, it also provides the inherent benefit of automatic incremental memory management. The system knows when objects are no longer needed and automatically recycles the memory they consume. Thus, Dynamic OOP solves the memory leak problem common to static OOP applications, and allows programmers to concentrate on the application problem domain without the distraction of building memory management systems.

Dynamic OOP also presents some compelling advantages for software maintenance. It eliminates the very long recompile times, sometimes taking hours or days, required by static languages just to change a few lines of code. The use of meta-data permits layering new functionality into deployed systems dynamically, without access to the underlying source code. As a result, systems can be upgraded electronically, by distributing new upgrade code. Dynamic OOP also enables developers to easily customize systems at user sites with minimal downtime.

Smalltalk

The two most prominent Dynamic OOP languages are Smalltalk and ANSI CLOS. Smalltalk, a first generation Dynamic OOP language, has become popular in the MIS environment because it offers significant productivity gains over SQL languages and COBOL and is simpler to use than C++. But while better than static languages, Smalltalk lacks features such as multiple inheritance and multi-methods that are critical for complex problems representation.

Smalltalk's performance is relatively poor, too. Originally designed for GUI exploration and not for building systems, Smalltalk is a "pure" object-oriented language, in which every datum is represented as an object. This means that even a simple floating-point addition - normally requiring only two or three machine instructions - takes as many as 200 machine instructions in Smalltalk due to the overhead of operating on any data type as an object. As a result, Smalltalk remains suitable primarily for creating front-end applications in client-server environments, not for large complex systems.

ANSI CLOS

The limitations inherent in Smalltalk inspired its creators at Xerox Palo Alto Research Center and other organizations to explore Dynamic OOP language further. This research lead to the development of CLOS, the Common Lisp Object System.

CLOS is a high-performance, second-generation Dynamic OOP language featuring automatic memory management, support for multiple inheritance, multiple methods dispatching and meta-object protocol. CLOS has its roots in Lisp, a performance-driven systems language used to create operating systems that has always been at the forefront of language technology. In the 1970s, for example, Lisp was enhanced with the latest "structured" control forms, something FORTRAN programmers at the time could only simulate by enforcing a discipline on the use of GOTO statements. Later, when Smalltalk demonstrated the usefulness of OOP, CLOS was added to Lisp through Flavors, an object system that went well beyond Smalltalk to include multiple inheritance and method combination. Today CLOS extends the object-oriented paradigm even further, and is the only ANSI standard OOP language.

CLOS is more than just a programming language. It comes with a very powerful, interactive development environment. Unlike static OOP languages, CLOS's late binding eliminates lengthy compile-link-run cycles, making it easy to define and modify program components and immediately debug them. CLOS's dynamic model allows classes and methods to be created or altered and objects to change class at run time, making it possible to write applications that change dynamically at run time, and allowing end users, third-party developers and even other applications to reconfigure a running application. Automatic and efficient use of multiple inheritance allows CLOS developers to easily apply highly specialized domain knowledge to generic classes.

A mixed-model or hybrid language, CLOS enables developers to declare objects such as floating point numbers as native data types. As a result, performance-critical elements written in CLOS can achieve performance comparable to that of C++.

CLOS has proven itself in commercial environments worldwide. In addition to Price Waterhouse and MIT, CLOS is being used by AT&T to develop sophisticated telephone switching systems, by GE and Pratt & Whitney to design jet engines, by Boeing to design the 777 and other aircraft, by Ford and Jaguar to design new automotive systems, by Motorola and Texas Instruments for manufacturing scheduling and integrated circuit design, and by Microsoft to create new and innovative applications such as Bob, the much talked about "social user interface."

The Promise of Dynamic OOP

The ability to dynamically change the behavior of an application enables rapid development times and quicker time to market because developers are able to alter the design to reflect their evolving understanding of the problem their application aims to solve. The rapid prototyping and user-evolved software development experienced by Price Waterhouse and MIT's AI Lab are just two examples. But even more dramatically, the ability to support dynamic change enables the creation of new types of applications not possible with static OOP.

Dynamic OOP lets software developers create a new breed of user-modifiable shrink-wrapped applications frameworks that can be easily transformed into custom applications at a customer site without the need for source code. Dynamic OOP is able to achieve the goal of high configurability and low cost by leveraging the power available on the desktop to produce software that can be significantly modified easily, and inexpensively, by the customer.

Dynamic OOP also opens the door to creating new markets for exciting new applications like intelligent agents. The future will be filled with applications that can dynamically reconfigure themselves, a capability that will prove to be critically important especially in networked environments and on-line systems.

By combining the attributes of dynamic, interactive languages with object-oriented programming, it becomes possible to dramatically alter the behavior of an application by "injecting" a very small amount of data while the application is running. This unique capability of Dynamic OOP is being popularized today by General Magic's vision of a new breed of intelligent network agents, compact pieces of code that modify the behavior of a server-based applications for a custom purpose by transporting new objects and methods across the network. Generated by a personal digital assistant (PDA), the agent attaches itself to an application running on the server, modifies its behavior to get the desired information and sends the result back to the PDA. Using Dynamic OOP reduces network traffic and allows automatic generation of the custom code, off-loading the compute- and data-intensive components of the system from the PDA while retaining true interactivity. This vision is already a reality for CLOS users such as Viewstar, a leading player in the burgeoning field of business document automation, where document flow becomes electronic over the network.

Compact software updates and bug fixes can be distributed over networks simply through email to thousands of computers without monopolizing huge amounts of valuable network bandwidth. Creators of on-line system front-end applications no longer have to anticipate every conceivable service provider's need, enabling creative looks and functionality for new services by simply uploading very small files containing the necessary behavior to the end-user application at connect time.

Made practical by the explosion in desktop computing power, dynamic object-oriented languages have come of age and are ready to take their place on the software developer's workbench. They are the next step in the evolution of ever more powerful, easier to use development tools necessary for creating software in a rapidly changing world.