In-Depth
Special Report: Enterprise Service Bus, Synchronicity in the Enterprise
- By Kathleen Ohlson
- October 1, 2005
Finding common ground for Abebooks’ IT group and its business operations
was like resolving the age-old feud of the Hatfields and the McCoys— it
just wasn’t happening.
“There was a series of problems dealing with the synchronization of the
IT group with the company’s goals,” says Jayson Minard, Abebook’s
CIO. “The history…was IT went in one direction and the company in
another.”
Abebooks is an online marketplace for books, with more than 77 million titles
and more than 13,500 member booksellers. It handles more than 6 million incoming
inventory updates, more than 2 million inventory searches, and more than 30,000
orders per day for more than 4 million visitors each month.
Application changes didn’t keep pace with business operations, and the
business logic was so scattered it slowed any changes. Forget about scaling
applications; any attempt crushed the application on top of itself, Minard says.
“We wanted to be ahead of the company for once,” he adds.
Minard investigated how other companies solved their messaging issues, eventually
selecting Sonic Software’s ESB to integrate disparate systems between
Abebooks, sellers, resellers and buyers. The Sonic ESB runs on top of SonicMQ,
which sends messages between business systems.
The side effect of ESB led Abebooks to implement service-oriented architecture.
Minard and his staff are currently working to make Abebooks’ inventory
system available to SOA, then order management, search and pricing.
The true meaning of an ESB
Enterprise service bus is the newest solution—and buzzword—for business
and vendors, and the latest addition to the enterprise architecture integration
market. ESB is an open standards-based messaging middleware that integrates
enterprise apps using different protocols and data formats to communicate via
XML and Web services. ESBs offer a core feature set: load balancing, failover
and portability.
Factoid
The Apache Software Foundation (http://wiki.apache.org/
incubator/SynapseProposal) launched Synapse, an opensource project to
develop a common, core building block for Web service infrastructure software,
including ESBs, Web services brokers, and Web services management products.
It’s a framework to intermediate between two or more Web services, allowing
for transformation and routing, and loose coupling between services.
Specific revenue for ESBs is hard to track, but the overall EAI market—
which includes application integration, middleware and portals—grew 2.5
percent to more than $6.2 billion in 2004, according to a recent Gartner report.
Cape Clear Software, Cordys, Fiorano Software, FusionWare, Iona Technologies,
PolarLake and Sonic are the main players in the ESB market, but some big guns—including
BEA Systems and IBM—are entering the territory. For example, BEA recently
released its AquaLogic Service Bus as part of its SOA strategy.
ESBs’ increasing popularity is related to yet another tech phenom—SOA. They
are a key component of SOA, providing the tools and runtime frameworks for developers
to encapsulate legacy apps and expose them as Web services. But ESBs alone don’t
build a SOA—a common myth, according to industry experts. (See related story,
“ESB: A Building
Block to SOA”)
Like any new hip technology, ESB has different meanings to different people,
generating confusion. Making the confusion worse is the variation among vendors’
offerings. For example, Sonic ESB is built on top of its SonicMQ messaging queuing
infrastructure, while BEA System’s AquaLogic bus is built on top of its
WebLogic application server.
Some vendors that call their products ESBs add BPEL support, orchestration,
management and other capabilities into the definition, says Jason Bloomberg,
an analyst at ZapThink. “So, if the message transport part of an ESB is
a proprietary bus infrastructure, then what really is the difference between
an ESB and the dolled-up porkers the EAI guys are selling? That’s the
$64,000 question.”
However, users have their own definition of ESB.
“We have what we believe is an ESB: Web services over messaging,”
says Abebooks’ Minard. “If they think it’s any more than that,
then we don’t have that.”
The book marketplace’s infrastructure includes an online e-commerce site,
an online bookseller site, five core servers and APIs for orders, returns, inventory,
search and e-commerce. More than 100 batch processes are chained together through
a central database. These applications used to consist of a “big monolithic
code base” that lacked flexibility, according to Minard. Sonic ESB and
SonicMQ now handle messages throughout Abebooks’ infrastructure. “We
think of [ESB] as a big messaging bus pumping a lot of data through,”
he adds.
Getting the message
The difference between ESB and EAI is quite evident to Mark O’Neal, project
manager for Tarrant County in Texas. The county is in the midst of implementing
a jail management system for the sheriff’s department and didn’t
want to deal with lots of .NET and Java coding. It initially didn’t focus
on ESB but gradually to meet the project’s goals.
“We have a much better appreciation for what an ESB can do for us,”
says O’Neal. “Figuring out how to plug that system into that one,
and doing things in between those two [systems] tend to take a life of their
own,” he adds. ESBs are configurable and designed for plug in, while traditional
EAI uses a hub-and-spoke model and requires custom development around it, he
says. ESB proved to be a much cheaper path; it will cost Tarrant County between
$125,000 from $150,000; whereas alternative methods ranged from $250,000 to
$300,000.
The jail management system will track when a prisoner is taken into custody,
including the reason the prisoner is in jail, criminal history, medical care
information and security classification. It will contain several hundreds of
gigabytes of data, not including mug shots and electronic documents. Prior to
using the system, the sheriff’s office managed this information with a
bunch of Excel spreadsheets and databases.
Sonic ESB will integrate multiple systems, including biometrics, the district
attorney’s office and Homeland Security, to the jail management system.
For example, if a person is arrested for DUI, a message will be sent to the
DA’s mainframe system that this person was arrested. In turn, the jail
management system will be notified if charges were filed, automatically pushing
this data to the Tarrant County Sheriff’s office. O’Neal expects
the entire system will be in place over the next 12 to 18 months, but declined
to give any more details.
“In the old scenario, a prisoner arrived to the backdoor of the jail
with a mound of paperwork,” he says. Tarrant County expects to see a “huge
cost savings,” though O’Neal declines any specifics, it’s
the equivalent of nine full-time employees. “Conservatively, it easily
offsets the cost in the first year,” he adds.
Master Bus Basics
According to Wikipedia, the online encyclopedia, the common characteristics of an enterprise service bus are:
- It requires the clear separation of message headers and message body
- It is usually operating system and language independent; it should work between Java and .NET applications, for example
- It (often) uses XMLand Web services to transport messages
- It includes adapter standards (such as J2C) for incorporating existing applications into the bus
- It includes support for asynchronous processing
- It includes intelligent, content-based routing
- It includes a standardized security model to authorize, authenticate and audit use of the ESB
- It includes transformation services (such as XSLT) between the format of the sending application and the receiving application, including the transformation of data formats
- It includes validation against schemas for sending and receiving messages
- It can uniformly apply business rules, enrichment of the message from other sources, splitting and combining of multiple messages, and the handling of exceptions
- It can conditionally route or transform messages based on a central policy
- It is monitored for message latency and other characteristics described in a Service Level Agreement
- It (often) facilitates “service classes,” responding appropriately to higher and lower priority users
- It supports queuing, holding messages if applications are temporarily unavailable
- It handles a “publish and subscribe” messaging model, including event handling
Code sanity
ESBs protect developers from some coding nightmares by independently supporting
operating systems and languages, so developers can keep years of existing code.
More importantly, ESBs provide a layer on top of enterprise messaging systems,
which allows developers to exploit messaging without writing additional code.
It was that coding option that led online mortgage lender Quicken Loans to ESBs.
Fiorano Software’s ESB spared online mortgage lender Quicken Loans from
creating more code to integrate its internal and external systems, including
existing record management and workflow platforms. Quicken Loans found integrating
up to five legacy systems troublesome since there was no way to guarantee message
delivery. “There was a lot of handholding of the system and integration,”
proving to be more difficult with limited staffing, says Warren Hampton, Quicken
Loans’ senior e-commerce architect.
The ESB, along with Fiorano Business Integration Suite, now allows Quicken
Loans to reuse codes, error handling, interactive debugging and rollback scenarios.
The online mortgage lender is slowly implementing Fiorano products into its
environment; Quicken Loans is processing a couple of hundred messages a day
and expects to double or triple that number in the near future.