In-Depth

SOAP eases JP Morgan Chase technology pain

• By Jack Vaughan
• April 1, 2002

The era of client/server computing was abundantly supplied with tales of marvelous systems that worked in the lab and failed upon rollout. Often, the rollout itself was the stumbling point. This reporter recalls a thorough multi-slide presentation concerning an advanced object system, which ended as the presenter sheepishly noted that the system was never built because it would become obsolete by the time it was loaded on all of several thousand insurance agents' desktops.

The story was not unique. And, of course, most system architects know the story quite well.

Awareness of the perils of rollout is at least one of the elements behind IT's interest in so-called Web services -- lightweight protocols that rely on the Web as a delivery mechanism. Worries about rollout issues was one of the key concerns that caused a team of developers at JP Morgan Chase to embrace new Simple Object Access Protocol (SOAP)-based Web services technology, despite their natural awareness that new technology has its own unique learning curve. The JP Morgan Chase development team succeeded because it was willing to innovate and, at the same time, to temper any bald desire to innovate with a pragmatic understanding of the system's ultimate goal.

Robert Greig, technical architect, and his colleagues at JP Morgan Chase's Global Research Content Division in Glasgow, Scotland, created a single-source financial data system for use throughout the company. Known as the Global Financial Database, or GFD, the system provides real-time calculation of financial items -- for example, earnings-per-share -- based on changing data from various sources and feeds. It supports sector-by-sector comparison and analysis for a team of analysts that truly does span the globe.

Technologies employed include SOAP, Ant build scripts, a multithreaded Java Server, a Sybase database and the open-source Tomcat server engine. Among a host of tools used were Sun's Forte for Java 3.0 Enterprise Edition IDE, Microsoft's SOAP toolkit, Rational's Rose modeling environment, the open-source JUnit tester and the CVS configuration manager.

Mergers are a way of life in industry now, and the financial arena is no exception. In the wake of significant mergers with Chase Manhattan and Robert Fleming, JP Morgan Chase's need for a single-source repository of financial data became very apparent.

The mergers required IT workers in the Global Research Content Division in the United Kingdom to take on the task of creating a system that now had wider, more global dimensions.

''Suddenly, we had all these new analysts all around the world,'' Greig said. ''But they needed a single source [of financial data].''

It was important that users of JP Morgan Chase financial data, he said, ''all had access to the same figures.''

End-user input was important. Therefore, end users were interviewed. (''In all continents!'' noted Greig.)

The existing systems to be migrated to the Global Financial Database consisted of a variety of relational databases. To illustrate the variety, Greig noted that ''one was a Microsoft FoxPro database with a manual process for updating.''

On their desktops, users employ specialized Excel spreadsheets. ''We needed to have full integration with Microsoft Excel,'' he said. This, then, was part of the challenge of the system's desktop rollout.

''It was a global desktop rollout,'' said Greig, ''and getting [for example] a CORBA desktop out to thousands of analysts would be a significant challenge.'' Moreover, due to mergers, IT could not guarantee the ultimate OS.

Also, there are firewall issues. Opening another port would have taken a great effort. Opening a port on a firewall is managed by a corporate infrastructure team, but it can add great complexity to a project.

''That was pain and suffering we wanted to avoid,'' he said. So SOAP was employed because it 'allows us to ensure connectivity.

''The Microsoft SOAP toolkit was fairly pain-free in many ways,'' he noted, then added, ''there was a certain learning curve. If we were doing it now, there are higher-level abstractions we could use.''

Noted Greig: ''We had to build some custom serializers to improve performance -- but we didn't have four-hour conference calls trying to get some obscure firewall open in the Far East.'''So one can surmise that using a new technology naturally requires some assembly, but it can be worth the effort.

What is Greig's estimation of the other tools used? ''The Forte tools were excellent because they integrate so well with other tools,'' he said. ''We were able to integrate Systinet WASP -- our SOAP implementation -- very easily, which was handy when getting up to speed with SOAP.''

He added: ''We definitely used a form of Extreme Programming [XP]; we did a lot of unit testing with JUnit up front. We didn't use the two-person [XP] teams.''

Meanwhile, Ant was used to provide a consistent build environment across Unix and NT, he said. ''We developed on NT boxes, but deployed and tested on Solaris. Ant meant that we had easy-to-configure build scripts that were fully portable across platforms,'' he said. ''Also, Ant, integrated with Forte, enabled us to do 'proper' builds from within the IDE.''

How might Greig advise other IT professionals on Web services? ''New technologies can be quite disruptive. If you're using a new technology, and are part of a team with interrelated projects, you have to show how the technology fits in. A lot of people didn't know how SOAP would fit in,'' he said. Looking back, Greig would like to have been able to spend more time informing other teams about how SOAP might work in their areas of expertise.

The JP Morgan Chase and other stories seem to indicate there is something behind the industry's push toward Web services. In the case of the Global Financial Database, ''pain points'' associated with a big system rollout were avoided, thanks in part to SOAP, the most evident representation of Web services to date. It may be the muddy little secret behind much progress, but avoiding pain has always been one of the big drivers of technical innovation.

[Editor's Note: In the printed version of ADT, the location of JP Morgan Chase's Global Research Content Division was listed as Edinburgh, Scotland. The technology office is actually based in Glasgow and has been corrected in this online version.]

Application profile:

Project: Global Financial Database (GFD)

Purpose: To provide an integrated, reliable source for financial information targeted at research analysts.

Benefits: More consistent data, easing retrieval of financial data, and freeing resources previously used to maintain different data sets.

Tools: Sun Forte for Java 3.0 Enterprise Edition IDE, Sun JDK 1.3.1, Sybase 11.9.2 ASE database, Systinet WASP 3.0 Advanced, Microsoft SOAP toolkit 2.0, Visual Basic 6.0 Enterprise Edition, CVS 1.11, Visual C++ 6.0 Enterprise Edition, Tomcat 4.0.1

Keane Report:

JP Morgan's project, the Global Financial Database (GFD), used many proven technologies, some ''bleeding-edge'' technologies, minimal staff and short time frames, yet managed to produce an important core system that a number of global applications depend on. The system is highly scalable, deployable and portable. It is of special note that a sense of practicality was displayed in both the selection of tools, methodologies and approach to the technologies. The pragmatic reasons given for the selection of tools and technologies, from diverse platforms and vendors, such as Forte for Java, Systinet's WASP 3.0 (SOAP Toolkit), Microsoft SOAP Toolkit, Sun's JDK and Tomcat, indicate a focus on achieving a result, as opposed to following the beaten path.

While the use of SOAP to overcome the expected firewall problems and enable rapid deployment is currently innovative, it is likely to become a standard procedure for many companies in the future. The use of SOAP as an abstraction, acting as the interface to the multithreaded Java Server, is something that the pundits have talked about for some time. It is excellent to see it become reality.

Team Leader: Neville Goedhals, Senior Principal Architect, Keane Inc.