Special Report: Enterprise Service Bus, Synchronicity in the Enterprise

Finding common ground for Abebooks’ IT group and its business operations was like resolving the age-old feud of the Hatfields and the McCoys— it just wasn’t happening.

“There was a series of problems dealing with the synchronization of the IT group with the company’s goals,” says Jayson Minard, Abebook’s CIO. “The history…was IT went in one direction and the company in another.”

Abebooks is an online marketplace for books, with more than 77 million titles and more than 13,500 member booksellers. It handles more than 6 million incoming inventory updates, more than 2 million inventory searches, and more than 30,000 orders per day for more than 4 million visitors each month.

Application changes didn’t keep pace with business operations, and the business logic was so scattered it slowed any changes. Forget about scaling applications; any attempt crushed the application on top of itself, Minard says. “We wanted to be ahead of the company for once,” he adds.

Minard investigated how other companies solved their messaging issues, eventually selecting Sonic Software’s ESB to integrate disparate systems between Abebooks, sellers, resellers and buyers. The Sonic ESB runs on top of SonicMQ, which sends messages between business systems.

The side effect of ESB led Abebooks to implement service-oriented architecture. Minard and his staff are currently working to make Abebooks’ inventory system available to SOA, then order management, search and pricing.

The true meaning of an ESB
Enterprise service bus is the newest solution—and buzzword—for business and vendors, and the latest addition to the enterprise architecture integration market. ESB is an open standards-based messaging middleware that integrates enterprise apps using different protocols and data formats to communicate via XML and Web services. ESBs offer a core feature set: load balancing, failover and portability.


The Apache Software Foundation ( incubator/SynapseProposal) launched Synapse, an opensource project to develop a common, core building block for Web service infrastructure software, including ESBs, Web services brokers, and Web services management products. It’s a framework to intermediate between two or more Web services, allowing for transformation and routing, and loose coupling between services.

Specific revenue for ESBs is hard to track, but the overall EAI market— which includes application integration, middleware and portals—grew 2.5 percent to more than $6.2 billion in 2004, according to a recent Gartner report. Cape Clear Software, Cordys, Fiorano Software, FusionWare, Iona Technologies, PolarLake and Sonic are the main players in the ESB market, but some big guns—including BEA Systems and IBM—are entering the territory. For example, BEA recently released its AquaLogic Service Bus as part of its SOA strategy.

ESBs’ increasing popularity is related to yet another tech phenom—SOA. They are a key component of SOA, providing the tools and runtime frameworks for developers to encapsulate legacy apps and expose them as Web services. But ESBs alone don’t build a SOA—a common myth, according to industry experts. (See related story, “ESB: A Building Block to SOA”)

Like any new hip technology, ESB has different meanings to different people, generating confusion. Making the confusion worse is the variation among vendors’ offerings. For example, Sonic ESB is built on top of its SonicMQ messaging queuing infrastructure, while BEA System’s AquaLogic bus is built on top of its WebLogic application server.

Some vendors that call their products ESBs add BPEL support, orchestration, management and other capabilities into the definition, says Jason Bloomberg, an analyst at ZapThink. “So, if the message transport part of an ESB is a proprietary bus infrastructure, then what really is the difference between an ESB and the dolled-up porkers the EAI guys are selling? That’s the $64,000 question.”

However, users have their own definition of ESB.

“We have what we believe is an ESB: Web services over messaging,” says Abebooks’ Minard. “If they think it’s any more than that, then we don’t have that.”

The book marketplace’s infrastructure includes an online e-commerce site, an online bookseller site, five core servers and APIs for orders, returns, inventory, search and e-commerce. More than 100 batch processes are chained together through a central database. These applications used to consist of a “big monolithic code base” that lacked flexibility, according to Minard. Sonic ESB and SonicMQ now handle messages throughout Abebooks’ infrastructure. “We think of [ESB] as a big messaging bus pumping a lot of data through,” he adds.

Getting the message
The difference between ESB and EAI is quite evident to Mark O’Neal, project manager for Tarrant County in Texas. The county is in the midst of implementing a jail management system for the sheriff’s department and didn’t want to deal with lots of .NET and Java coding. It initially didn’t focus on ESB but gradually to meet the project’s goals.

“We have a much better appreciation for what an ESB can do for us,” says O’Neal. “Figuring out how to plug that system into that one, and doing things in between those two [systems] tend to take a life of their own,” he adds. ESBs are configurable and designed for plug in, while traditional EAI uses a hub-and-spoke model and requires custom development around it, he says. ESB proved to be a much cheaper path; it will cost Tarrant County between $125,000 from $150,000; whereas alternative methods ranged from $250,000 to $300,000.

The jail management system will track when a prisoner is taken into custody, including the reason the prisoner is in jail, criminal history, medical care information and security classification. It will contain several hundreds of gigabytes of data, not including mug shots and electronic documents. Prior to using the system, the sheriff’s office managed this information with a bunch of Excel spreadsheets and databases.

Sonic ESB will integrate multiple systems, including biometrics, the district attorney’s office and Homeland Security, to the jail management system. For example, if a person is arrested for DUI, a message will be sent to the DA’s mainframe system that this person was arrested. In turn, the jail management system will be notified if charges were filed, automatically pushing this data to the Tarrant County Sheriff’s office. O’Neal expects the entire system will be in place over the next 12 to 18 months, but declined to give any more details.

“In the old scenario, a prisoner arrived to the backdoor of the jail with a mound of paperwork,” he says. Tarrant County expects to see a “huge cost savings,” though O’Neal declines any specifics, it’s the equivalent of nine full-time employees. “Conservatively, it easily offsets the cost in the first year,” he adds.

ESB: A Building Block to SOA

Master Bus Basics

According to Wikipedia, the online encyclopedia, the common characteristics of an enterprise service bus are:

  • It requires the clear separation of message headers and message body
  • It is usually operating system and language independent; it should work between Java and .NET applications, for example
  • It (often) uses XMLand Web services to transport messages
  • It includes adapter standards (such as J2C) for incorporating existing applications into the bus
  • It includes support for asynchronous processing
  • It includes intelligent, content-based routing
  • It includes a standardized security model to authorize, authenticate and audit use of the ESB
  • It includes transformation services (such as XSLT) between the format of the sending application and the receiving application, including the transformation of data formats
  • It includes validation against schemas for sending and receiving messages
  • It can uniformly apply business rules, enrichment of the message from other sources, splitting and combining of multiple messages, and the handling of exceptions
  • It can conditionally route or transform messages based on a central policy
  • It is monitored for message latency and other characteristics described in a Service Level Agreement
  • It (often) facilitates “service classes,” responding appropriately to higher and lower priority users
  • It supports queuing, holding messages if applications are temporarily unavailable
  • It handles a “publish and subscribe” messaging model, including event handling

Code sanity
ESBs protect developers from some coding nightmares by independently supporting operating systems and languages, so developers can keep years of existing code. More importantly, ESBs provide a layer on top of enterprise messaging systems, which allows developers to exploit messaging without writing additional code. It was that coding option that led online mortgage lender Quicken Loans to ESBs.

Fiorano Software’s ESB spared online mortgage lender Quicken Loans from creating more code to integrate its internal and external systems, including existing record management and workflow platforms. Quicken Loans found integrating up to five legacy systems troublesome since there was no way to guarantee message delivery. “There was a lot of handholding of the system and integration,” proving to be more difficult with limited staffing, says Warren Hampton, Quicken Loans’ senior e-commerce architect.

The ESB, along with Fiorano Business Integration Suite, now allows Quicken Loans to reuse codes, error handling, interactive debugging and rollback scenarios. The online mortgage lender is slowly implementing Fiorano products into its environment; Quicken Loans is processing a couple of hundred messages a day and expects to double or triple that number in the near future.

“It allows us to put wrappers around the systems and get ready for SOA and newer systems,” Hampton says. “Anything we do going forward will be SOA.”

Sidebar: “ESB: A Building Block to SOA
Case Study: "ESB Helps Stratus Clear Road Partners"
Chart: “Where to Catch the Bus