In-Depth

Renew, don't rewrite, legacy systems

• By Peter Fischer
• December 1, 2001

Even though the dot.com threat appears to be over, there is little relief for traditional brick-and-mortar organizations that need to be "e-business-enabled" to remain competitive in today's rapidly changing economy. The harsh reality is that the backbone of the modern IT environment is a diverse set of technology and applications. In addition, an estimated 70% of all corporate data and processing resides in mainframe-based legacy systems. The greatest challenge facing most organizations is unlocking the processing assets stored in these legacy systems to enable their use in new e-business initiatives.

To achieve this "e-business nirvana," organizations must integrate with their legacy systems systematically and strategically to become more efficient and effective. Creating integration points allows the enterprise to utilize existing processing assets more efficiently. Packaging these integration points as reusable assets enables the enterprise to open up new business opportunities.

IT leaders are faced with the most significant challenge in this process: figuring out how to do all of this. The good news is that there is no shortage of tools and technologies available to help solve these legacy integration woes. The bad news is that the real challenge is deciphering which combination of tools and technologies is the right one for your mainframe environment. This article presents a framework for legacy integration and provides some starting points for products that can put your organization on the right path.

Companies are very concerned with balancing the business demand for rapid time-to-market with their ability to take advantage of their sometimes more than 20 years' worth of installed applications in a structured, managed and controlled way. This focus on leveraging legacy assets is not surprising considering the "run-the-business" nature of these "big iron" systems and the huge investment that was recently made to make them Y2K-compliant.

With Y2K in the past, companies' sights are now set on leveraging legacy assets to build new business functionality. Companies are particularly focusing on legacy integration to offer some form of relief. It is important to note that legacy integration is not a rewriting process, but an extension process. I have been evangelizing this concept for the past several years with clients looking to solve their integration problems. Setting legacy integration into the context of a renewal or extension program is a key starting point.

To remain competitive, companies are turning to leading-edge "e-nablement" technologies such as Java, XML and integration brokers to help them create a new generation of applications. These composite applications are a combination of existing and new data and processing assets, which are created by integrating functionality from one or more existing applications using one or more integration techniques.

Legacy extension offers the best solution to delivering these composite applications by providing the foundation for combining new functionality with existing processing assets. According to a Gartner Inc. estimate, by the year 2003, 80% of application development organizations will leverage some form of legacy extension technology to accelerate the benefits of composite applications without the associated expense of application replacement.

The biggest challenge involved in legacy extension is understanding how to take traditional task- and function-oriented systems that reside on the mainframe and make them an integral part of a process-driven and customer-facing IT processing environment. Let's face it—today's terminal applications, including 3270 and 5250 apps such as the common order-entry application, will not cut it in a highly competitive landscape.

As e-business initiatives evolve, the requirement for a comprehensive integration-centric program that provides the foundation for deploying mission-critical systems becomes vital. Companies that take a tactical, "band-aid" approach to solving their legacy integration woes by attacking the symptoms and not the problem—replacing the old technology by slapping a new HTML-based user interface over existing "green screen" interfaces, for example—are creating tomorrow's legacy today.

Enterprise Application Integration (EAI)-based solutions offer the most comprehensive approach to creating legacy extension and integration points. Organizations that treat the creation of these integration points as the end-game are selling themselves short on opportunities to leverage the investments made in their integration projects. Thinking strategically, legacy integration becomes a journey, not a destination.

The foundation of a comprehensive e-business legacy integration program is a three-pronged approach: It is based upon a five-layer integration solutions architecture model, the packaging of legacy functionality into reusable "eaiComponents," and a unified technology foundation known as the Enterprise Integration Middleware (EIM) platform. This combination allows fragmented business processes, such as "Add Customer" or "Modify Customer"—that are spread across multiple mainframe-based applications—to be centralized into a single composite application (see Fig. 1).

Figure 1: Legacy integration

Legacy integration utilizing composite applications and components simplifies access to mainframe application functionality.

The EAI Solutions Architecture Model (see Fig. 2), provides the blueprint for e-business integration solutions. This five-layer model is similar to the J2EE model in its separation of presentation and business logic. What makes this model unique and valuable for legacy integration is the explicit integration layer that connects business components and services to back-end systems.

Figure 2: EAI Solutions Architecture Model

EaiComponents allow developers to package discrete legacy functionality as reusable assets within the business logic layer. Isolating the business logic in this way "repackages" functional and task-oriented applications as services and business components that can be reused across multiple integration initiatives.

Within the business logic layer, these components are architected in a component-based framework that provides key extension points. This makes the framework easy to extend as business needs evolve and underlying technologies change. More importantly, this framework allows the provisioning of existing functionality and business processes as services that are executed and coordinated in the space of new application server technology, and then provided to partners and customers as well-formed Web services.

The EIM, which provides the underlying platform of products and technologies that enable integration, ties all this together. The EIM consists of multiple products that enable full integration across the IT environment. Figuring out how to evaluate and select the appropriate tools to implement the EIM is the toughest part of the job.

Strategies for legacy renewal
There are many dimensions to legacy integration that need to be considered when creating an EIM based on appropriate tools and technologies. Two of the most important criteria are the levels of intrusiveness and integration.

A non-intrusive solution allows integration to be achieved without touching the legacy environment. In most environments this might be required because of the immutable, or hands-off, attitude most firms take toward their legacy assets.

An intrusive approach requires modification to the current host environment, ranging from adding new enablement products to the execution environment—called environment invasiveness—to altering legacy applications, which is called application invasiveness. Environment-intrusive solutions are preferable because they do not require the existing application code to be modified. The advantage of the environment-intrusive approach is its ability to create an "in-place" integration layer that can leverage the scalability and manageability of the existing host environment.

Legacy applications can be integrated at four levels: User Interface, Application, Method or Data (see Fig. 3). Data-level integration is by far the most prevalent and straightforward approach to legacy integration. With this approach, components packaged as data objects can access legacy data utilizing JDBC or ODBC—all you need is a copy of the data model and SQL.

Figure 3: Integrating data at four levels

However, this approach poses severe risks because in most apps, business logic and data manipulation reside in the application code and not in the database. Going directly to the data bypasses this functionality. This could force you to rewrite these rules and then maintain them in two different places, which would defeat the purpose of integration. In addition, data integrity logic is bypassed, which could lead to key data integrity issues.

User Interface (UI)-level integration targets the current presentation layer and user interface of legacy applications by intercepting datastreams. This typically involves screen scraping, which many observers perceive to be a primitive and inelegant approach. However, sometimes it is the only option, considering the monolithic nature of most of the current legacy installed base, which is often characterized by intermingled and tightly interwoven presentation and business logic that requires a major rewrite of the code base to separate.

In reality, UI integration involves two different approaches: the first is GUI extension through on-the-fly emulation of screens as HTML, and the second is wrapping screens as objects. GUI extension is a proven and widely used approach for first-generation legacy integration solutions. It is also a cost-effective and rapid solution to re-facing existing terminal-based applications. However, all it does is re-face the legacy apps. The end result is an HTML interface over older green-screen applications where screen-to-screen navigation is still driven by logic embedded in the underlying application.

These solutions are tactical in nature and do not offer the features and capabilities that strategic solutions do, including scalability, performance, reliability and reusability. Most significantly, their lack of componentization leaves a significant gap in reusability, which yields low-impact integration solutions. In reality, GUI extension tools provide only a "patch" to the current problem, and these solutions will quickly become tomorrow's legacy applications.

One of the reasons this approach falls short is its focus on integrating content and user interface over integrating transactions and processes. While content integration is important, the real return on investment can be found in leveraging legacy applications for their "run-the-business" processes and transactions. The better solution, which yields reusable components, is the "screens as objects" approach. Here, the functionality and navigation among screens is captured in well-formed eaiComponents that have well-defined service interfaces that essentially capture complex navigation paths as a single method and associated data.

EaiComponents capture end-to-end transactions as they are currently viewed for the application and its users. They are created by a tool that resembles a terminal emulator, which captures the executable model or description of the interactions required to communicate with the host program to perform business transactions. Each model contains all of the information required to perform discrete operations on the legacy application, including screen recognition, keystrokes required to navigate between screens, regions on screens where input and output fields are placed, error handling and data. During the execution of an eaiComponent service interface method, the model is "played back" to the running terminal-based application, which is supplied with data from the eaiComponent.

Method-level integration provides access to legacy functionality as a set of service methods supplied via a transactional interface. Access to functionality is provided via an interface made up of a set of well-defined transactions and related semantics.

For example, CICS provides method-level access to transactions within applications via the External Call Interface (ECI) programming interface and access to screen functionality via the External Presentation Interface (EPI). The ECI enables method-level integration by providing access to CICS transactions from eaiComponents. ECI calls a CICS program and passes it a commarea. The EPI enables user interface-level integration by requesting that CICS install a terminal session on its behalf. In a J2EE environment, eaiComponents would be Enterprise JavaBeans (EJBs) that wrap these transactions and provide access to them via remote interface methods.

Application-level integration is provided via a programming interface or API that allows external applications to access functionality and data within a legacy application. Because this approach requires an existing programmatic interface to be available, it is typically the most difficult level to achieve because APIs are rarely written to provide access to CICS or IMS applications.

During the past several years, messaging has emerged as a key technology for legacy integration at the application level. Instead of exposing interfaces as a set of API methods, this approach utilizes message formats to provide the interface for applications.

Messages are formatted and sent via a Message-Oriented Middleware (MOM) product to the existing mainframe app that has been made "message-aware." The app processes the message and a reply is then sent back to the requester with the required data. EaiComponents use a messaging API, preferably a standard like JMS for eaiComponents captured as EJBs, to access the functionality of the MOM product and process messages. IBM's MQSeries product provides enterprise-class messaging capabilities.

Integration products provide some relief
There are a plethora of products on the market that can provide integration across a spectrum of levels and approaches. The challenge, as always, is wading through the choices and making the appropriate selections.

The Jacada Interface Server (JIS), from Atlanta-based Jacada Software, provides a simple and easy-to-use non-invasive approach to user interface-level integration. JIS supports componentization for both Java and COM, which gives developers flexibility in the way they can package legacy components. Another key feature is its ability to host multiple 3270 and 5250 applications simultaneously. In addition, JIS exposes legacy and new business logic as XML transactions, which gives developers flexibility in the way they tie the information together.

CrossPlex, an environment-invasive product from Oklahoma City-based SofTouch Systems, offers an alternative. Besides its GUI extension capability, CrossPlex provides screens-as-objects functionality by remodeling existing legacy assets.

Remodeling combines information from multiple screens, eliminating the screen-to-screen navigation imposed by 3270 applications. An easy-to-use scripting language allows developers to build components that drive legacy transactions or extract data from legacy applications. An automatic screen-object generation facility provides one-stop maintenance. Because CrossPlex is a host-side product, it provides componentization independent of the client platform and does not require the presence of a middle-tier application server.

InnerAccess, from Toronto-based InnerAccess Technologies, is another user-level integration product providing a different approach to legacy integration. Instead of operating on the 3270 data-stream, it intercepts a 3270 application before it becomes a 3270 datastream and turns it into XML. It accomplishes this by operating at the BMS level and not the 3270 level. One advantage to using InnerAccess is that the resulting components are not field-dependent—repositioning the fields on the screen will not impact the generated integration code. The resulting XML can then be packaged as a message that is sent to an integration environment, providing true support for the "externalize-then-integrate" approach.

Verastream Host Integration Server, from WRQ, Seattle, provides leading-edge integration user interface-level mining tools. Host Integrator is a set of object-oriented development tools that allow developers to interact with host applications and data from an integrated development environment. Legacy assets are modeled as objects that are created through a graphical point-and-click interface. The underlying model, stored in a repository, contains all the meta data required for legacy application transversal and navigation. Components can then be created in a variety of formats, including COM and EJB, and deployed in appropriate application server environments.

Application server vendors are also getting into the legacy integration game by providing adapters for their products. For example, BEA Systems Inc., San Jose, Calif., provides the WebLogic Java Adapter for Mainframe (JAM) with its WebLogic Integration (WLI) platform. JAM facilitates access to IBM CICS and IMS mainframe applications from Java application components running on the BEA WebLogic Server (WLS). The eGen COBOL facility makes life easier for Java developers by automatically generating Java source code from COBOL copybooks.

JAM provides bi-directional, online request-response integration between Java apps and IBM CICS or IMS apps. Because it provides a familiar environment for Java app developers and mainframe programmers, no special skills are needed to develop eaiComponents and composite apps.

JAM allows EJBs, servlets and Java classes running on WLS in the business logic layer to initiate online requests to CICS or IMS programs, and receive results online. In addition, it allows CICS and IMS programs to initiate online requests to EJBs running on WLS and receive results.

Integration brokers from companies such as SeeBeyond Technology Corp., Monrovia, Calif., Vitria Technology Inc., Sunnyvale, Calif., and WebMethods Inc., Fairfax, Va., provide built-in adapters for integrating their environments with mainframe assets. These tools offer general integration capabilities across a range of technologies and platforms and most provide basic support for accessing CICS via the commarea. Some also offer support for capturing screens as objects. Because almost all the tools on the market today support Java, components captured by specialized tools can be deployed in integration broker environments in a straightforward manner.

IBM offers a variety of technologies and tools for accessing legacy functionality across its host environments. Some of these are available in the base product or as a support pack, which can reduce costs. Products such as WebSphere and VisualAge provide an almost completely out-of-the-box solution because they include these technologies.

IBM provides native support for Java on the mainframe via JCICS, which allows new applications to be written in Java and executed within CICS. In addition to the benefit of not having to write new program logic in COBOL, JCICS provides integration benefits. It can be used to wrap existing host assets, enabling new J2EE eaiComponents to speak in their "native tongue" to legacy components.

One of the first Java-CICS enabling products was the CICS Transaction Gateway (CTG). CTG provides Java classes, called CICS Transaction Gateway classes, that enable Java code to issue either ECI or EPI calls to CICS. The CTG converts these calls into real ECI or EPI calls that CICS can process.

IBM's MQSeries, the leading message-oriented middleware product on the market today, provides message-based integration to legacy assets stored in all IBM host environments, including CICS and IMS. MQSeries for MVS provides a CICS interface that allows CICS programs to read and write MQSeries queues, allowing components to interact with CICS programs via MQSeries messages.

If the CICS application is written to use MQSeries for input and output, no application changes are required, which yields a non-invasive integration solution. If it is not designed to use MQSeries, an environment-invasive approach is necessary; a front-end CICS transaction needs to be written to process messages and communicate with the existing CICS program via the commarea. JCICS can be utilized to write the new code. In either case, these transactions cannot be 3270-based.

Non-invasive access to 3270 transactions is provided via the 3270 Bridge, which intercepts CICS terminal commands and passes control to a user-written exit that provides input from and output to an alternative source such as MQSeries or Java.

Access to IMS transactions and data is accomplished through a combination of Java enablement tools and products. IMS Connector for Java is a collection of Java classes that provides a Common Connector Framework (CCF)-compliant interface to IMS transactions using IMS Connect. The CCF allows IBM's VisualAge Java development environment to generate code for these integration components in a standard way utilizing Enterprise Access Builders (EABs). IMS Connect provides standard technical connectivity between TCP/IP clients and IMS Transaction Manager subsystems. Using this approach, Java eaiComponents are built using VisualAge EABs that create all the code necessary to implement integration components. Because these components are J2EE-compliant, they can be used across multiple applications.

Capturing legacy functions and tasks as eaiComponents and integrating them within a component framework allows composite applications to be created. These composite applications provide single-point access to key processes and procedures that are spread across multiple and disparate legacy applications. Take, for example, the Add Customer process depicted in Figure 1, where customer information is spread across three systems. To accomplish a complete Add Customer transaction, three separate and distinct mainframe applications have to be accessed, which can lead to expensive and error-prone manual data entry across multiple applications.

An eaiComponent framework groups eaiComponents together into an architected environment by providing the ability to execute the transaction from a single point. In Figure 4, a component framework based on the popular Model-View-Controller (MVC) design pattern illustrates how to combine the eaiComponents, or model components, with a transaction eaiComponent or controller component. The controller component combines the three model eaiComponents into a single transaction, while the view component provides the interface to the composite application by providing a simple service interface to related business transactions.

Figure 4: Combining eaiComponents with a transaction eaiComponent or controller component

A controller component combines the three model eaiComponents into a single transaction, while a view component provides the interface to the composite application by providing a simple service interface to related business transactions.

This significantly reduces the complexity of application development and fosters reuse. Packaging legacy assets as eaiComponents shields application developers from the underlying integration enabling technologies and related complexities of the EIM. The model eaiComponents can be created once and used across a variety of transactions.

In addition, new capabilities for transaction processing or the addition of new applications that must be integrated are isolated to a single point. If a new Add Customer application is introduced—perhaps because of a merger or acquisition—a new eaiComponent can be created that wraps the applications and can be added to the framework with no impact on the existing composite application. This is truly an integrate-and-extend approach.

The story continues
The effective use of eaiComponents does not stop with their development and initial deployment. While deploying eaiComponents within frameworks enables reuse, it does not guarantee it. The way to prevent eaiComponents from becoming tomorrow's legacy is to enable them for reuse. This requires a structured and disciplined approach that enables evolution and utilization across the IT environment.

A component reuse program, backed by a component repository and supporting development tools and environments, must be put into place to reap the return on investment eaiComponents promise. It is only through reuse that the effort and cost of repackaging legacy assets can be realized.

As we have seen, tools and technologies to aid in legacy renewal and extension span a wide range of capability and applicability. These include tools such as screen scrapers, transaction APIs, messaging products, integration brokers, application servers and code generators. An important point to remember about creating EIMs is that there is no single product or technology that can solve all your integration problems; rather you need an integrated technology platform that is architected for flexibility and extensibility. Typically, point-integration products, in combination with an app server and integration broker environment, provide the foundation for a solid EIM.

The triumvirate of eaiComponents, an EAI solutions architecture and an underlying EIM provide an "externalize-and-integrate" approach that enables legacy assets to be re-formed as components in an integration environment and presented to new composite applications as eaiComponents. Packaging legacy-integration points as reusable eaiComponents in an architected framework that executes over an EIM activates true legacy extension and renewal.