In-Depth

Whither TP Middleware?

• By Deborah Melewski
• August 1, 2001

Before the advent of Internet-based e-commerce, corporate transaction-based systems were mostly large back-office apps running on mainframe computers. Today, transaction-oriented middleware aims at providing more open, flexible solutions based on Internet technology. But some observers wonder aloud whether transaction middleware is headed the way of the abacus, because Web application server technology seems to handle all that middleware does in addition to the development and deployment of large-scale transactional applications.

So far, corporate IT operations continue to buy traditional transaction processing systems, such as CICS, Tuxedo and Encina, as well as some newer offerings to process both Internet and non-Internet transactions. The traditional systems have been changed markedly and still support more transactions than any existing technology. But what of the future of traditional TP technologies as application servers and other technologies gain many of their features? Many industry experts—and even some executives at traditional TP monitor suppliers—expect the technologies to be absorbed into new integration and middleware products.

Whatever happens, the technology is still vital for a variety of mission-critical applications. For example, Visa USA, Foster City, Calif., runs the largest private payment processor in the world using traditional TP technology. Owned by more than 21,000 member banks, the Visa network processes more than 35 billion transactions each year and runs at peak rates of 3,500 messages per second, according to Visa USA. That number grows 20% to 30% each year. To date, the message volume has doubled every three years.

Visa USA also has one billion cardholders and is accepted at 16 million merchant locations worldwide. The network settles transactions worth about $800 billion each year. To handle that volume, and to uphold Visa's rather well-known pledge to be "everywhere you want to be," the company chose to extend its decade-old mainframe-based facility by building out a new network and application infrastructure.

"We wanted the ability to process all forms of electronic payment," said Sara Garrison, vice president of technology at Visa USA. "We also needed to be completely scalable, and we needed to maintain our standards of 100% availability and reliability. We needed to process at a rate of 10,000 messages a second, handling 100 billion transactions annually," she added. "That equates to about a trillion dollars in payments annually."

Visa's new switching system is intimately tied to open technologies. "We needed a message-oriented middleware component around which we could build," said Garrison. "We were looking for the middleware to include: a message queuing capability, 100% reliability, a lot of flexibility to grow our architecture, support for all types of payments, performance to standard, data integrity, scalability and extensibility, interoperability, maintainability and a product that was proven in the marketplace."

Visa USA chose to go with Tuxedo, the TP monitor born in the laboratories of AT&T many years ago, and now from BEA Systems Inc., San Jose, Calif. "Tuxedo's reputation is that it is best-of-breed in message-oriented middleware, so we wanted to look very closely at that product," said Garrison.

In the end, Visa USA chose Tuxedo for its performance and high volume processing statistics required for its specialized, high-intensity app. "It could support all of our functional requirements and it is highly flexible, which enables a lot of customization," said Garrison. "It has the ability to handle large-scale systems. We chose it because it's a mature product that's been around in one flavor or another for about fifteen years."

The maturity and reputation of the Tuxedo product, as well as BEA as a company, was important in this decision. "At the end of the day, you want to go with technology that will let you sleep at night, knowing that they are going to be there in the morning and that they are going to evolve as you go forward," said Garrison.

According to Garrison, the benefits of using new open Internet technologies in transaction processing have already been realized. Visa Direct Exchange has run through its pilot, testing and tuning stages. "As a result of Visa's new messaging service, which is a high-availability cluster solution set comprised of Sun Servers, Oracle databases and Tuxedo middleware, we're going to be driving out to the types of performance measures we have been looking for," noted Garrison.

"The underpinning of the Internet is flexibility and openness," she said. "These qualities allow you to eliminate proprietary solutions at each end of the electronic dialogue, which means you're able to more rapidly deploy products. With the elimination of proprietary solutions, you're able to more flexibly mix and match information and payment services, and reduce the high costs of maintaining proprietary connections."

Looking toward the future, what will Visa USA expect from BEA and Tuxedo? "As we deploy Direct Exchange, our new enabling infrastructure, we will also be looking at driving out 'Net-centric product solutions," said Garrison. "What I mean by that is multi-channel delivery using open technologies. This will allow us to provide customized products that can flexibly mix and match payment and information services to our end users—which are the banks—and ultimately, the consumer.

"To build out this system, which will be integrated with our switching technology that is provided by Tuxedo today, we will be operating at a scale that will be mind-boggling. Remember, we have 1 billion cardholders."

Working on that kind of scale, Garrison expects Visa USA will have to use the Web as a high production environment. Part of that 'Net-centric architecture will be tied in with other Web products offered by BEA such as the WebLogic Server suite. BEA has played an active role in all phases of the Visa project, including design, development, test and deployment, noted Garrison.

Traditionally, middleware was conceived as a layer of software that helps integrate applications running on different computers or different applications running on a single computer. Clearly, most of today's e-commerce and e-business applications involve multiple applications running on multiple systems. Thus, the integration, communication, messaging and transaction capabilities that were traditionally associated with middleware are still needed.

John Kiger, director of product marketing at BEA, said many of those capabilities are included in a product set—application servers—that adds many extensions to support Internet and World Wide Web technologies. "Middleware, for the most part, has been subsumed by a new generation of technology called the application server," said Kiger. "The application server continues to provide all of the facilities traditionally associated with middleware, such as access to databases, access to legacy systems and system-to-system messaging, but adds much more, such as all of the Web application capabilities."

Kiger sees customers increasingly turning to application server and full e-business platform technologies because they deliver a richer set of functionality that covers the full spectrum of needs for developing and deploying e-business applications today.

What does this mean in terms of Tuxedo's future as a more generalized TP middleware technology? "We continue to sell Tuxedo as a product," Kiger said. "We have many high-profile customers who are building large mission-critical applications using Tuxedo, which supports traditional programming languages and paradigms such as Cobol, C, C++ and CORBA."

"We also continue to keep Tuxedo fresh and moving forward with the integration of new technologies as needed," said Kiger. Last year, BEA added support for public key encryption, which is the basis for Internet security, and support for digital certificates. XML support has also been added.

"Our customers can continue to build on Tuxedo, with the assurance that it is integrated with not only their existing systems but also their new systems, which may be built on BEA's WebLogic Server, which is our Java platform," said Kiger. This enables the customer to integrate applications built across multiple platforms.

Another approach
Not quite ready to implement the full Web application server approach, Chicago-based Quikorder Inc. chose to go with the Cache transactional middleware solution from Intersystems Corp., Cambridge, Mass.

Quikorder.com is a business-to-consumer e-commerce site that targets online ordering of restaurant food. Quikorder's initial target market is pizza delivery restaurants. Site visitors can order from 500 stores throughout the United States. The company's primary client is Domino's Pizza.

Quikorder.com allows customers who sign on to the site to find the nearest restaurant and place orders for delivery or pickup using customized menus specifically set up for each restaurant.

"We take that a step further than most e-commerce sites do," said Marc Asher, CTO at Quikorder. "Instead of providing the order to the store in the form of an E-mail, or having to manually call the store or use the IVR Voice Response System to call the store, we transmit the order straight into the local store's point-of-sale computer system. This method allows us to ensure the order received is accurate and to provide information such as estimated delivery time to the customer."

The Quikorder site has been live for a little more than two years. Architecturally speaking, the site is not exactly a typical three-tier application. "We're closer to the two-tier model, but we are moving more towards three-tier," said Asher. "The application and the database currently coexist. With the Cache database, we have that kind of flexibility."

To provide this service, Quikorder needed a database that could handle relatively small amounts of relational data with fast transaction processing. Cache from Intersystems fit the bill.

Among Cache's best features, according to Asher, are its rapid application development capabilities. "We were able to quickly prototype systems and various modules of our system, and then get them into live running systems very quickly," he said.

Quikorder looked at other relational database technologies as well, but the combination of Cache's rapid application development capabilities and its performance, particularly with the small pieces of relational data that the Quikorder system handles, brought Cache to the forefront.

Quikorder users need to be dynamically assigned to a store location, then given the ability to pull up a table with pricing for a particular store. The order should be dynamically re-priced each time. Users may also need to pull up a previous order history, submit that information and then have it transmitted to the remote store within about one minute. To handle that volume of separate relational data from several different tables, the underlying database's performance was critical.

Quikorder built its own internal monitoring system. In addition to the Cache application, the company runs Microsoft IIS as the Web server on an NT platform. To help ensure 100% uptime, Quikorder went with the Marathon Assured Availability Server from Marathon Technologies Corp., in Boxborough, Mass.

Future expectations
Cache has evolved into an object-oriented-type language, to which Quikorder will be transitioning. "At that point we'll be moving to the three-tier architecture, with a strong separation between our relational database, our application code and the client, which in this case is someone accessing the site with a Web browser," said Asher.

Asher said his firm also plans to utilize a variety of technologies to build a powerful transaction processing system. "We'll also be looking to take advantage of some significant advances Intersystems has made in terms of Web integration to the database in what they call their Cache Server Pages [CSP] technology," he said.

As broadband and other high-speed connections become more popular, Quikorder will also look to create an enhanced user experience. This would involve adding Java plug-ins and multimedia capabilities.

Right now, Quikorder primarily uses strict HTML coding. The company does minimal JavaScripting. "This approach has given us very fast responses as well as compliance across multiple browsers including WebTV," explained Asher. "We wanted to ensure compatibility across the board."

Quikorder has been able to achieve the core functionality required. "Many of the middleware-type tools do not apply to what we're doing," said Asher, and the Intersystems technology has met the conditions.

In the future, Asher notes that the company may move to add more tools for application development. "At that time, middleware technology may become more of an issue," he said.

According to Paul Grabscheid, vice president of Strategic Planning at Intersystems, not all Cache customers deploying in a three-tier architecture choose to bring Web application server technology into the mix.

"This is one of those issues about which some people are very passionate and some people are very blasé," he said. "We do have some customers working with Web application servers. In my experience, they tend to be customers who have adopted Java as their development technology."

Grabscheid likened this to the Sun/Java and Microsoft camps. "We saw this as another religious war that nobody was going to win, but people in both camps could be found doing things successfully. So, we don't push our customers in either direction."

There are typically two reasons for using Web app servers, noted Grabscheid. The customer may have a heterogeneous environment, with a lot of database systems and a lot of different systems and technologies that need to talk to one another. In this case, the Web application server will serve as the connection point, because it can speak to lots of different types of technologies.

Beyond that, people in the Java world want to take the Java components in their app and keep them from burdening the Web and database servers. People may tend to put them in the middle of three tiers, or in some cases even four tiers, sitting between the Web and the database servers.

Looking to Cache's future, Grabscheid is seeing continued growth in the high end, and people trying to do bigger things as systems get bigger. He also expects to see increased focus on security and privacy.

Intersystems is very active in the health care market, where a lot of attention is given to medical privacy on the Web. "We have this often conflicting environment, where you want to make it easier to get at medical information for people authorized to do that; but you also want to make it harder to get at the medical information for everybody else," said Grabscheid.

For example, medical emergencies do not always happen at convenient times. Doctors may need to access a patient's medical records in the middle of the night. Hospitals are therefore starting to use the Web in a secure way so that doctors can access that type of information from home.

Yesterday's CICS, today's WebSphere
Transaction processing has always been IBM's core competency. Going back 20 or even 30 years, IBM's CICS and IMS products were usually the leading transaction monitors worldwide, often by a wide margin. Building on that heritage, IBM's transaction processing product set for the Internet is WebSphere, a family of middleware products that includes technologies such as CICS and MQSeries.

With the advent of e-business, IBM aggregated the transaction processing technology capabilities of CICS into its WebSphere Application Server line. In the enterprise edition of WebSphere Application Server is the TX Series, which is Distributed CICS—basically the non-390 CICS—as well as Encina, the distributed object-oriented transaction monitor originally built by Pittsburgh-based Transarc and acquired by IBM several years ago.

The object transaction monitor that is part of Encina was rewritten in the years following its acquisition, and was incorporated into WebSphere Application Server Advanced Edition in 1999. "We had CICS, IMS and Encina—all very strong, market-leading transaction monitors back in the 1980s," said Scott Hebner, IBM's Director of Marketing for WebSphere. "As we moved through the 1990s and focused more on our e-business strategy, we integrated those technologies into WebSphere. In WebSphere we now have a fully J2EE-based Web application server environment that is also a transaction monitor."

As a longtime leader in the corporate transaction processing business, observers say IBM was poised to take its core competency in transactional capabilities to the Internet. "As the Internet became a platform to which people built, it provided the ability for businesses to integrate their operations and to reach new customers whether they were end users, consumers or other businesses," said Hebner.

To help customers leverage and extend current assets to the Internet, IBM deployed a series of transactional connectors to its Web application servers. This allows customers to take CICS or IMS—or different transaction monitors on the back end—and make their transactions available on the Internet. According to Hebner, the connectors, coupled with IBM's VisualAge family and WebSphere Studio development tools, can ease the transition by putting an Internet front end on the transaction systems.

Yet another challenge is moving from the procedural-based transaction environment to a Java-based environment, or from a proprietary-based to an open standards-based environment. "Our recommendation to our customers building for the Internet is to build to the J2EE technology, which includes transactional capabilities," said Hebner. "We provide a very robust transactional engine under the covers of that."

What of the Encina middleware? Just as BEA Systems moved to build Tuxedo into its WebLogic Server technology, IBM has incorporated the Encina transaction monitor into its WebSphere application server.

Said IBM's Hebner, "We encourage customers to build on the Java application server because it has a lot of the Encina technology. We still offer Encina as a procedural-based distributed transaction monitor, as part of WebSphere Enterprise Edition. Customers would use that if they have existing apps running under Encina, or if they are not ready to move to Java yet, or if they want to write in C++."

Transactional capabilities are clearly becoming more important than ever with the Internet. However, this means moving from a centralized transactional environment or a distributed environment that could be controlled to a much more open, distributed transactional environment that is both asynchronous and synchronous.

"Transactions over the Internet require you not only to be able to manage a distributed set of transaction monitors, but also a transaction that can now be very longstanding," said IBM's Hebner. "If any part of the transaction does not commit, you need to roll the whole thing back, so you must look at this as composite transactions. The whole transaction does not get committed until everything has occurred."

With the Internet, much of the focus has also shifted to Web application servers as the transactional platform. But clearly, industry experts say, the powerful technologies of the veteran transaction processing systems will still be required as business moves to the Web. The question is how that technology will be used. Few observers believe the technology will remain in standalone systems as new integration options move into the corporate mainstream. Customers will continue to have many choices, which observers say should provide the ability to build on specific requirements.