In-Depth

Can components solve integration conundrum?

• By John D. Williams
• July 1, 2001

For IT managers, the need to integrate has renewed an interest in component-based development, whose promise to become a developer panacea a decade ago was crushed due to overly complex technologies, a lack of interest among first-line engineers and a slew of other reasons, industry observers say.

Rapidly changing markets, emerging new business opportunities and customer expectations are all driving forces behind what business gurus call business integration. These factors are working their way into IT organizations, radically affecting budgets and projects. For IT organizations, the demand for business integration translates into the need to integrate systems and the components that built them. Thus, integration is not an option; it is a critical factor in delivering products to market or applications to the business more quickly—and it has renewed IT interest in CBD, observers say.

Developers of corporate systems no longer have the option of developing most systems from scratch. Most legacy systems must be integrated with next-generation Web-based business offerings to give users almost unlimited access to critical corporate data. Even in the context of a single project, there is a need for integration of various technologies. For example, a single application can incorporate components supplied by a development environment, components purchased from a variety of sources and internally developed components. Managing and integrating this maelstrom of components is a complicated task. As we will see, integration is more than something that takes place using an Integrated Development Environment (IDE).

Why is integration an issue?
Why is integration such an issue for developers? It is certainly true that the forces demanding business integration have filtered through the corporate pipeline to the software developers, but that is not the only reason. Three factors—the needs to develop applications faster, cheaper and better—are still driving the adoption of components. Businesses are demanding quicker deployment of applications while refusing to pay the ever-increasing development and operational costs of building the software from scratch. Component development and its complement—reuse—are seen as key tools in the bid by IT organizations to meet these challenges. Many IT organizations are adopting CBD because of, executives say, the renewed promise that buying, building and reusing components can save significant amounts of both time and money. Experience shows this is true, but experience has also shown that implementing significant CBD and reuse projects is not as simple as some observers maintain.

Sam Patterson of Kennesaw, Ga.-based ComponentSource, a supplier of packaged components, estimates that a total of 20,000 to 40,000 components must be available from suppliers before CBD can become a viable option for large IT units. For the component market to achieve critical mass there must also be a sufficient number of components available to allow many applications to be built by simply assembling existing components. Component enthusiasts concede that only one-third the ideal total of packaged components is currently available. While the number of packaged components has consistently doubled annually during the past few years, IT developers must still build a significant number of components, and integrate them with legacy applications and packaged systems to implement a CBD stratagem—a truly difficult task. This in turn drives the need to integrate purchased and homegrown components.

In a similar fashion, we can think of a company's reusable components as an internal marketplace. This internal marketplace raises many of the same questions as the external market. How does one find the desired components? Will they fit the company's needs? How will users learn to use them? The answer is systematic reuse. Systematic reuse is a key issue in integrating components. Time will not be saved reusing a component if it cannot be found. Integrating it with others will be impossible if a user does not know how it works. A user cannot reduce costs if everyone goes out and buys their own favorite version of a component. Managing a company's components is an important part of integration, time and cost management. Systematic reuse can make developers more effective, create less frustration and keep costs down. It needs to be a part of the development and integration processes.

Another factor in integration is the availability of appropriate component technology. It generally does not help when the components a user wants to buy are in a technology different from the one in which they are working. Currently, COM and Java define the component mainstream. COM components significantly outnumber their Java counterparts, but that is changing. While the number of COM components continues to double every year, the number of Java components is tripling. Once the dust clears and developers begin to understand the implications of .NET, expect to see an increase in components oriented toward this technology. There are only smatterings of components commercially available in other technologies.

There are two options for integrating incompatible components and systems. The first is to adopt a bridge tool. These are typically tools for bridging between COM, Java and CORBA. Unfortunately, there are challenges with these tools. The technologies do not have one-to-one mapping, so bridging from one technology to another can be problematic. These tools may or may not be a good solution, depending on the way the technologies need to communicate.

The second choice for integration is to use a component technology-neutral communication mechanism, which can take several forms. If the goal is to focus on Web technology, HTTP may be used for communication. Most component technologies are HTTP-conversant. Another option would be to exchange information in a neutral format and choose XML as the lingua franca. These approaches can also be combined, and a wire protocol such as the Simple Object Access Protocol (SOAP) could be used to facilitate communication over HTTP using XML. Messaging technology such as MSMQ or MQSeries provides even more control over the communication. The messages themselves may be XML or even SOAP payloads.

Component integration is an issue for tool vendors as well. Vendors want to make it easier for developers to integrate components and build systems. Of course the vendors would prefer developers do that using their tools. Developers benefit because vendors are starting to compete with each other to help them find and integrate components. An example of this is the recent agreement between ComponentSource and Santa Clara, Calif.-based WebGain. WebGain is providing access to ComponentSource's component marketplace from within the WebGain IDE. It is part of the new Application Composer visual construction tool WebGain recently announced. The tool is designed to allow developers who need components to use in their systems to be able to see quickly what is available in the marketplace, download it and try it—all from within their development environments. It benefits developers, but it presents additional challenges to IT organizations that are trying to manage components and costs.

Other tool vendors view the integration of components in a different light. Some recognize the challenge organizations have in managing the rapid growth of components in their development shop. An organization may have purchased or homegrown components, but making sure it has the right ones and ensuring they deliver value to the organization is a challenge no IDE can address. Organizations need people, processes and tools to manage the systematic reuse of components across projects and organizations. There has been a flurry of activity around repository tools this year that reflects a growing awareness of this need. San Diego-based Aonix recently purchased what had been the Select Component Manager from Princeton Softech, Princeton, N.J. Cary, N.C.-based Castek RBG's reUSE-IT repository and reuse center automation tool was successfully deployed in several organizations. And in early 2001, both Santa Rosa, Calif.-based IntellectMarket and Cleveland-based Flashline announced new tools in this area.

These differing points of view illustrate just how complex the issue of integrating purchased and homegrown components can be. There are issues within a selected technology, which include finding components to integrate as well as the challenges of integration itself. There are issues in integrating components across technologies. There are challenges in selecting tools that will help with the integration. There are also challenges in managing the components that will be integrated. Let's look at these issues in more depth.

Component technology issues
There are a number of component technologies available to developers. Java has JavaBeans and Enterprise JavaBeans. Microsoft has COM, which is morphing into .NET. Lesser-known technologies such as Islandia, N.Y.-based Computer Associates' COOL:Gen are also available. One value that component technologies bring to developers is a set of standards for developing and integrating components. With both COM and EJB, there are standards for component behavior in the infrastructure. For example, with EJBs, there is a set of container-related behaviors that all EJBs have in common. The same is true with MTS-based COM components. These standards facilitate common component integration.

These technologies also focus on the component interface. The emphasis on the interface, while hiding what is behind it, is a distinguishing characteristic of component technology. There are two advantages to this characteristic. First, it allows developers to focus more on solving business problems without becoming bogged down in technology issues. Good component systems will be architecture-driven. Second, architecture-driven designs will make it simpler for developers to identify components that may be purchased for inclusion into the system. The architecture helps developers identify the interfaces needed as well as the components that match the needed interface. Without the architecture focus, it can be difficult to create well-structured components. This in turn makes purchasing components that are a good fit much more difficult. In short, well-defined interfaces are a developer's best friend when it comes to integrating components.

We see the effect of well-defined interfaces in application frameworks. In their article "Designing reusable classes," (Journal of Object-Oriented Programming, June 1998) Ralph Johnson and Brian Foote said, "A framework is a set of classes that embodies an abstract design for solutions to a family of related problems." I have always liked the definition used by former IBM subsidiary Taligent Inc., which reads, "Frameworks are not simply collections of classes. Rather, frameworks come with rich functionality and strong wired-in interconnections between object classes that provide an infrastructure for the developer." The most important point is that an application framework can save between 50% and 90% of the development effort on an application. This can represent a significant cost savings. Well-defined interfaces are the mechanism for tying in the customizations and additional functionality.

Most frameworks are expanded through one of two types of interfaces. The first approach is to create "hot spots" in the framework where variability is allowed. These are driven by the business architecture. The parts of the framework that remain unchanged across applications can be frozen. Areas that are expected to vary from application to application can be set up as hot spots where developers can plug in their own specific code. Hot spots are usually created as abstract classes or templates. The creator of the framework will define the interfaces of the hot spot and the set of generalized responsibilities it must fulfill. Application developers are responsible for plugging in their own application-specific replacements. Often framework developers will provide a number of concrete classes that can be used or serve as examples for customizing hot spots.

The second type of interface used to expand frameworks is a hook. Hooks are often virtual methods in classes that application developers are expected to override with their own application-specific behavior. Abstract classes can also be used as hooks. Both of these interface techniques can be combined as the framework developer sees fit. Regardless of which is used, these well-defined, technology-compatible interfaces are the simplest way of integrating a purchased framework and homegrown components.

Another way to interface to a framework or other component technology is to use a technology-neutral communication mechanism. Messaging and RPC mechanisms have been the traditional approach to this type of communication. SOAP is one of the newcomers in this area. SOAP is a lightweight protocol for exchanging information. It provides a means of invoking methods on servers, services and components. SOAP is not defined to be bidirectional and is typically used in one-way messaging. However, it can be used bidirectionally. It is an XML-based protocol and is typically used in conjunction with HTTP and HTML. If an application can talk HTTP and parse XML, it can probably use SOAP as a distributed communication mechanism. Notice that because it can work in a distributed environment, it can provide an alternative to technology bridging tools. It lets developers loosely couple disparate technology environments regardless of their location.

Managing components
Before developers can integrate with purchased components, they must be able to find them, know what they do and understand how to use them. Furthermore, enterprises need to manage the cost of components and insure that they get full value from using them. This leads us into the issues of how to successfully manage components for integration.

What happens when the 20,000 to 40,000 components the marketplace needs are actually available? How will developers find the ones they need? What happens to an organization that has even a fraction of those used internally? How will it avoid unnecessary duplication and cost? How will it find and reuse the ones it does purchase? Answering these questions is important for the integration of purchased and homegrown components to become a key method of building applications.

There are multiple initiatives underway to address the issue of finding necessary components. Rational Software Corp., Cupertino, Calif., has sponsored the Reusable Asset Specification (RAS). This specification is aimed at providing a common means of describing components so developers can find them. IntellectMarket uses a software component capsule to provide a universal descriptor that works across technologies. ComponentSource has created a reusable component specification that they have used as input to RAS. All of these approaches provide ways to categorize components so developers can find them and compare them meaningfully. These various approaches should converge during the next year.

Meaningful categorization is one part of managing components, but it is not enough. Businesses need to manage the cost and deployment of components. What happens if every developer goes out and buys their favorite version of the same component? Costs will go up and reuse will decline, producing an effect that is the exact opposite of the goal for using components in the first place. A managed repository can help with these issues. It is not enough to simply have a repository. If everyone simply contributes whatever they want to the repository, it ends up resembling a landfill. With a managed repository, components can be selected for their fit in the enterprise architecture and their reuse can be promoted to projects that need them. This requires people and reuse processes to be added to the development mix. This approach will let a company achieve real value for their investment in a component.

As mentioned earlier, with the growth of the component market, companies see the need for repositories to manage their components. Castek RBG's reUSE-IT is a repository for all types of assets (not just components) and automates several reuse processes. The Aonix Component Manager provides a basic component repository. Flashline's Component Manager Enterprise Edition (CMEE) is a basic component repository with two additional features: it supports some reuse metrics and provides a tie to Flashline's public component marketplace. IntellectMarket's IntellectMachine is a component repository with advanced capabilities for categorizing components. It also has e-commerce, ASP and portal capabilities. Regardless of which repository is chosen, allying the tool with people responsible for managing the components it will contain is necessary.

So what are the key things to remember in integrating purchased and homegrown components? First of all, adhere to component standards when building components. Using an architecture-driven approach will ease the process of finding purchased components and integrating them into the system. As the number of available components grows, consider acquiring components that are not core to the business. Why build something when it can be bought and deployed for less? Also consider purchasing application frameworks. These can add tremendous value by providing 50% to 90% of needed functionality for applications. The hot spots and hooks frameworks need to be compatible with the technology base. If the integration with different technology or distributed systems is necessary, use a component technology-independent communication method such as messaging or SOAP. Finally, managing an acquisition and deployment of components to achieve real value from their use is needed. Tools such as repositories can be a real aid in this, but also needed are people to manage the reuse processes.