In-Depth

5 stories about XML and integration

XML has found a growing role in application development since its inception five years ago. It has proved to be a useful way for describing and handling data, and, in application integration particularly, its role has become pivotal. XML has demonstrated its value in the eyes of a number of practitioners, despite the occasional headaches that come with working with what is new. Sometime benefits have accrued to newer Web services standards that stand on the shoulders of XML.

Is five years a short time or a long time? For some, it has gone by pretty quickly. For others, the slow road to a new tier of industry-specific XML standards has been arduous. Conversations with XML users indicate that much has been accomplished, and that much more work is needed.

Because XML provides a universal way of describing data, it has multiple diverse applications. After all, gathering, pushing, pulling, parsing and exploiting data is mostly what IT is about. More than any other technology, XML, when it appeared, set off little light bulbs in the minds of application developers of various ilk.

Just as XML means many things to many people, integration means much to many. Depending on the data, the job or just the ingrained prejudgments of the practitioner, the XML integration task is likely to be tackled from a different angle. At any stage, overhead can become an issue.

XML is far from monolithic, and it plays out in different ways in different application development organizations. Where data or documents set the architectural paradigm, XML is treated one way. Where the transaction is the main modus operandi, XML is treated differently. Practitioners with a grounding in EDI, B2B, EAI, object databases, application servers, messaging middleware, publish-and-subscribe middleware, business process management or data warehousing come to XML and integration with different ideas as to what can be accomplished.

Often, developers’ backgrounds are in more than one of these areas; that also affects the way they apply XML “glue.”

One thing about XML: It works better as it becomes more widespread. But conversations disclose that large companies that are well along in XML conversions cannot always mandate that smaller companies in their supply chain switch to XML.

This buttresses earlier projections made by IDC, which in 2002 estimated that Web services would roll out to the large enterprises (of which IDC says there are about 10,000 in the U.S.), then the mid-sized enterprises (120,000 or so, says IDC), and then the small enterprises (counted at 2 million) over the next decade.

Historically, integration has been difficult. Often it has depended on the “connector person,” that quiet individual in the corner who understood an interface that pre-dated everyone else on the development team. Talks with practitioners show that XML tools for mapping data sources are beginning to automate some of the domain-specific knowledge of the connector person.

The question, five years into this latest paradigm shift, is: How much has XML helped? ADT recently talked with a selection of XML implementers to see where XML stood. They talked of tools (which seem to be improving); standards (none complained about too many XML standards); and, as said, easier mapping (at least compared to the days when every connector was unique). We will let their stories do the talking.


1. Trade-Ranger drills for oil

QUOTE: Randy Clark, Trade-Ranger: “I think that we’ve only begun to tap the value of XML in trading partner transactions. Ultimately, it will allow us to economize and optimize business processes, and take some of the friction out of the supply chain. We do need to lower the barriers to adoption.”

The impact of XML is substantial, but its spread to mid-size and smaller companies has not been fast enough to suit Randy Clark, communities vice president at Trade-Ranger Inc. in Houston. Clark’s and Trade-Ranger’s goal is to create a marketplace for electronic procurement in oil, gas and related industries using CIDX, xCBL and PIDX standards.

Trade-Ranger has been on the cutting edge of middleware and XML since its inception. The company has implemented software from B2B specialist CommerceOne, XML integration expert webMethods and, more recently, Contivo for XML schema construction and catalog creation. All these software types tap XML to some extent as an alternative to earlier Electronic Data Interchange (EDI) standards that often required the services of value-added networks to accomplish transactions.

Clark’s role in this is to manage Trade-Ranger’s relationships with various members of its community. As such, he takes close part in petrol industry standards work. And he is responsible for content management of the catalog data that lies behind online petrol exchanges. Standards work can be time-consuming, but Clark and Trade-Ranger benefit from their association with industry groups that can agree on XML catalog versions of common oil and gas forms and data types.

PDIX transaction standards were preceded by work on EDI, and that is a benefit as the oil groups move to XML. But on the whole, moving from EDI to XML is not an easy undertaking. That is especially true for companies in the field that are not on the top rank of revenue.

''XML is starting to have some impact on [our] industry, although it hasn't happened as quickly as [we] thought it might,'' said Clark. ''Only the largest of the companies in the industry can afford it. It is not at the level below the multimillion dollar companies.''

Standards work is paying off for integration, however. ''Over the last year and a half, we developed a set of XML schemas primarily around the 'procure-to-pay' transaction sets,'' said Clark.

While webMethods middleware forms the basic interchange medium, Trade-Ranger used Contivo software for the canonical modeling. ''Being a marketplace, you need to have an enterprise model so you can map into and out of it,'' said Clark.

While webMethods middleware forms the basic interchange medium, Trade-Ranger used Contivo software for the canonical modeling. “Being a marketplace, you need to have an enterprise model you can map into and out of,” said Clark.

“We need to do a lot of translations of XML vocabularies and formats. Contivo within webMethods helps us create a single master model of XML to help us do the translations to and from the various XML standards that we have to support,” he said.


2. OneSource reports

QUOTE: Pete Frese, OneSource: “Before XML, you’d have to go in with a program specific to the data to do the processing. Now software allows the company to lay out sources and targets and do mapping.”

It has always been something of a struggle when two companies have needed to talk in a digital format. But XML is easing the struggle. “XML provides a standardized mechanism when you start interfacing between companies,” said Senior Software Engineer Pete Frese of OneSource Information Services Inc., Concord, Mass.

OneSource is an online business information service that consolidates data feeds gathered from many provider sources. XML makes it possible for subscribers to mix and match OneSource feeds, and to embed them within enterprise apps such as Siebel. OneSource uses Data Junction data-source mapping software to integrate -- some might say aggregate -- its input resources. “It’s been a good tool for processing the data that moves in an organization,” said Frese.

Before XML, said Frese, you would have to go in with a program specific to the data to do the processing. Now software allows OneSource to lay out diverse sources and targets and do mapping.

“Data Junction is more of a configuration tool, in my view, as opposed to a coding tool. It does the simple mapping. If the mappings you need are more complex, you can still do that fairly quickly and simply,” he said.

“We just completed an effort with financial reports on 30,000 companies,” said Frese. “It is quite a wide range of reports. It all comes in to us now in XML. We use Data Junction as an integration tool to read in what we want, and process what we want. We use Data Junction to do our lookups and to add other values, and to put it out where it can be accessible.

''Their interface lays out the schema. Or you can read it in so you don't have to code the schema. The mapping conversion program doesn't have to hit each row. It does the whole file and then reads out row by row into memory,'' added Frese.

It is a good thing to do as much in-memory as possible, he suggests. A useful emerging XML standard in OneSource's estimation is XBRL, which is gaining attention just as financial reporting itself gains renewed attention.


3. Quantum leaps in

QUOTE: Hal Render, senior staff program analyst, Quantum: “The one thing that needs to come along for us is more accepted document types for the messages people pass back and forth.”

“XML is a nice, general-purpose, intermediate language that we can translate our data in and out of,” said Hal Render, senior staff program analyst at Quantum Corp., a maker of tape backup equipment headquartered in San Jose, Calif. The XML integrations he pursues today seem to us to be an outgrowth of EAI, but may be directly described more as a form of B2B.

The good news for Quantum is that XML has general purpose and is working. But XML as practiced today has room for improvement. Said Render: “The one thing that needs to come along for us is more accepted document types for the messages people pass back and forth.”

Render and crew came to use IBM’s Business Integrator software via the back door. Quantum had selected EAI star CrossWorlds software for integration efforts, and then became an IBM customer when CrossWorlds was purchased by Big Blue.

He said his group looked at Tibco, SeeBeyond and Active (now part of webMethods) before selecting CrossWorlds. “We felt it had a nice mix of good, high-level design support and implementation. We did a bake-off, gave a spec, gave a week, and judged on how they did it and how they worked with [Quantum].”

There are two sides to XML, noted Render. One is XML in its base format; the other is the XML that underlies Web services. His work to date is firmly on the first side, as there has not been “a specific business driver” for Web services at this point.

Render reminds us that it took a long time to develop EDI as we know it, and XML must achieve what EDI has achieved. “EDI is big and complicated,” he said. Meanwhile RosettaNet XML standard alternatives have taken a while to develop.

Looking at Web services, Render said it appears to be a good way to handle technical issues that used to be accomplished by more language-specific RPCs.

''Web services still require that you have a designed interface. It's just a transport mechanism. The hard part about that is that you have to design and build to a fairly robust and flexible API, and it raises the general issue that, if the API gets customized, you face the old point-to-point problem,'' he said.

Render reminds us that it took a long time to develop EDI as we know it, and XML must achieve what EDI has achieved. ''EDI is big and complicated,'' he said. Meanwhile, RosettaNet XML standard alternatives have taken a while to develop.

“We originally planned long-term on using technology like RosettaNet. I think the RosettaNet people will get there. But it is a significant cost to switch-in tool sets. The thing EDI people have is a lot of time on the ground doing this,” he said.


4. Swiss Re-insures efforts

QUOTE: Motti Goldberg, head of IT for e-solutions project, Swiss Re: “XML by itself is a technology. It’s also to some degree an extension of a way of thinking [that may allow] the separation of functionality and semantics. But XML is only a tool.”

“XML is only a tool. When I hear vendors say we can do interoperability because we use XML, I think that is bad,” said Motti Goldberg, head of IT for e-solutions at Swiss Re. He went on to say that the semantic understanding of problems may be missing, despite the use of XML. “You may have only the syntax,” he said.

Goldberg’s project at the Swiss re-insurer makes use of XML software from Sonic Software. Swiss Re came to use Sonic by way of Excelon, the XML database maker Sonic acquired last year. Among the XML projects at Swiss Re is one that aims to support as much of the re-insurance value chain as possible over the Internet.

That means starting by bridging the gap between the company and insurance brokers, said Goldberg, and then trying to “automate the processes as much as possible.” Among those brokers are many who are unable to support XML conversion themselves at this juncture, making it necessary for Swiss Re to provide that functionality.

A favorable aspect of the Sonic solution, Goldberg said, is its ability to handle business processes. It is important for Goldberg to separate the process flow rules from actual processing. “XML helps there,” he said. This and other XML projects at Swiss Re have benefited from XML forms standardization efforts of the Accord insurance standards organization, said Goldberg.

Full use of XML may be a case of “back to the future” for Swiss Re. “Our first release was fully XML, to the degree where everything between internal modules flowing was XML,” Goldberg said. Even documents for presentation were XML.

“We were 100% XML. This is not where we are now. We learned where it makes more sense, and where it made less sense,” he noted. Speed and memory issues were also part of this equation.

“Between the services, the flow is still as XML documents, but the interface has been changed to be Java. So we have less transformation in the system.”


5. Johnson Controls environment

QUOTE: Byron Hill, director of system and technology marketing, Johnson Controls: “It’s all about the APIs. This gives us a clean set and standard way to develop apps, even when they are not resident on the same box.”

XML value for buildings systems giant Johnson Controls Inc. has to do with .NET, Web services and open APIs. “It brings us into the applications world. It’s a nice infrastructure but it is what you do with systems that makes a difference,” said Byron Hill, director of system and technology marketing at the Milwaukee, Wis.-based company, as he discussed XML Web services, which, for his MetaSys system undertaking, rely heavily on Microsoft .NET technology.

MetaSys is a building management system. Its initial purpose in life was to control the environment within a facility or building. Such systems have typically been pretty “hardwired,” commented Hill, but that is changing as fairly sophisticated functions, including security, get hooked into building systems.

The use of XML and Microsoft .NET Web services as a new type of abstraction layer above the basic building system will allow Johnson Controls and third parties to offer valuable services, potentially creating new businesses. “The beauty of it is that it allows us to work very easily with firewalls,” said Hill.

Of course, there was a learning curve here. “For some, it was a crash course,” Hill said. “Web services are still kind of in infancy in terms of being put into apps. Most are used in closed networks rather than open nets. We worked with Microsoft and other companies to learn more about them, and then continued to track the technology.”

Johnson Controls will publish a set of Web services, and, in effect, tell outsiders that this is the way to talk with their system. “So we don’t have to make changes on our side of the API,” said Hill. “That is the value Web services bring. In the past, we have provided our own protocol for talking with our system. Now we can say, ‘When you want to integrate on the system level, that’s the way you talk with us.’

“Now it’s not a year development cycle to build a system that talks with our system,” he noted.

Most   Popular