Columns

EAI: A business issue for IT

• By John D. Williams
• January 1, 1900

Enterprise Application Integration (EAI) is a booming business. Wintergreen Research estimates that $500 million was spent on EAI tools in 1999, that $900 million will be spent in 2000, and that EAI spending could reach $7.3 billion by 2003. Wintergreen also estimates that half of the world's 1999 IT labor budget was spent on developing custom EAI code and functionality.

EAI is a business issue, not a technical one. Rapidly evolving markets, new business opportunities and changing customer expectations are driving businesses to abandon their old stovepipe approaches to IT systems. The creation of new customer- and supplier-facing systems requires an integrated view of the business, which can be a real challenge for IT shops with isolated legacy systems. One helpful tool for understanding EAI is a framework that will help you comprehend the breadth of EAI, the type of integration you need to do and the business value of your approach.

Let's begin by defining EAI. That alone is a challenge. Many confuse EAI with generic middleware. There is some overlap, but the terms are not synonymous. Let me share two possible definitions with you:

"EAI solutions are software products that completely or partially automate the process of enabling custom-built and/or packaged business applications to exchange business-level information in formats and contexts that each understands." — Ovum, Enterprise Application Integration: Making the Right Connections

"It is essential to keep in mind that it is the chaining together of discrete transactions, in the form of a business process, from one application to the next that constitutes EAI." — Cherry Tree & Co., Extended Enterprise Applications

....Note that EAI is not just shipping data back and forth. EAI is also about processes, the flow of information and the rules that govern that flow. This is not to say that just exchanging information between systems is not EAI. The definitions help us understand that EAI is broader in scope and more inclusive. Cherry Tree & Co. defined an EAI hierarchy that provides an interesting framework for understanding these definitions. I'll give you my own version of their hierarchy.

The hierarchy has four levels: Transportation, Data Transformation, Business Rules and Business Processes. At the lowest level of EAI is the Transportation layer, which is data exchange. This is where messaging and much of what people think of as middleware fits in. In fact, GartnerGroup defines five application communication models — Conversational, Request/ Reply, Message Passing, Messaging-and-Queuing, and Publish-and-Subscribe — that fit in this layer.

Conversational communication is seen as programs interacting with one another directly. This is usually done through low-level APIs. Request/Reply is similar to Conversational communication, but is based on a single request and reply. This may be done with an RPC mechanism. Message Passing is a simple one-way communication. It is usually unblocked, which means the sending program does not need to wait for a response. This powerful capability allows systems to work together asynchronously. Messaging-and-Queuing is a store-and-forward messaging model with a middle stage. Messages are not sent directly from one system to the next. Instead, messages go to an intermediate queue that stores them until they are requested by another system. Publish-and-Subscribe is an informed method of routing data to interested systems. Essentially, systems subscribe to information provided by another system. When a system provides new information, that information is sent to all systems that have subscribed. This approach has the great benefit of minimizing the number of connections between systems.

Many companies attempt to integrate systems only with the Conversational or Request/Reply models. After a few of these style integrations have been developed, your system architecture begins to resemble a plate of spaghetti. The other models allow more flexibility and robustness, but require additional middleware to support them. It is in this layer that you find many EAI and middleware vendors.

The next layer up in the EAI hierarchy is Data Transformation. This layer is often found where architectures integrate systems at the data level. Data is moved from one database to another, and usually transformed in the process. This is what database vendors are talking about when they describe their products as EAI solutions. Other tools, such as those used to scrub data and populate data warehouses, also fit in this category.

The eXtensible Markup Language (XML) and its associated standards could play an important role at this level. For example, XSLT-based translations can simplify the transformation of data formats between systems. XML can also play a role in semantic and marshaling format transformations. The challenge will be to control how XML is used. XML could lead to greater data complexity as businesses struggle to adopt common standards for information interchange.

The third layer in the EAI hierarchy is Business Rules. This layer is where most full-blown EAI products reside. Essentially, the EAI tools in this layer use business rules to control what happens in the two lower layers. For example, an EAI tool such as Mercator will work with a messaging system such as MQSeries to route and transform information flowing from one system to another. Mercator may perform the data transformation itself based on defined business rules, or it may leave that to other middleware. It may also route information based on its publish-and-subscribe capabilities.

The level of sophistication and control you have with these tools varies from product to product. Many tools claim they operate in this layer, but matching their capabilities to the EAI hierarchy helps you understand where they really belong. Many true EAI tools have capabilities that overlap layers. GartnerGroup did four case studies with companies using this type of EAI tool. On average, using this level tool reduced development costs by 33% and maintenance costs by 66%, a dramatic improvement.

One weakness many of these tools have is their inability to control the flow of a single piece of information across multiple systems. For example, they will do a great job of sending information from System 1 to the next system or systems in line (System 2). What they often don't do is control the flow of that transformed information from System 2 to System 3 in the same context. For most EAI systems, each flow is a discrete transaction. They can't see the entire context of a business process and all of the systems information must pass through before it reaches its final destination.

This brings us to the fourth and final layer: Business Processes. This layer of EAI provides for the management of complete business processes. It also covers the last missing piece of system integration: enterprise workflow. This is the layer that can add the greatest business value. Tools in this layer will allow you to reshape and mold processes to match changing business needs.

....Imagine that you've used component technology to create clean interfaces to your legacy systems, effectively turning them into reusable components. Imagine that you've also developed your new E-business systems as reusable components. Enterprise workflow will allow you to assemble and reassemble these components as needed to create new processes. As you move up the layers, from Transportation to Business Processes, you improve the flexibility and robustness of your business processes. You reduce the complexity you need to embed in any given application, reduce intersystem complexity and shorten development time.

Barriers to EAI adoption

So why doesn't everybody move in this direction? Eventually they will, or they will be overtaken by their competition. But there are factors that can make it difficult to adopt EAI. The first is the initial cost of tools. As you move up the layers, initial costs increase. It is true that each step up can add significant business value, but many companies find it difficult to swallow the initial $350,000 to $1,000,000 investment needed in the higher layers.

Secondly, new tools require new skill sets. They can also require changes in a developer's architectural perspective. Some developers are more comfortable writing code at the lowest layer and don't want to be bothered learning new tools. This is why so many companies end up with spaghetti architecture as they integrate their systems. Training and an understanding of true business value can help address these problems.

While many tools claim to be the answer to Enterprise Application Integration, no one tool meets all needs. The EAI hierarchy serves as a framework for understanding levels of application integration, tool appropriateness and EAI business value. Try using this framework the next time you evaluate an EAI project or tool.