In-Depth

Web services and the mainframe

• By Richard Adhikari
• June 1, 2004

The idea of using platform-independent Web services technology to provide access to back-end mainframe applications is picking up speed. Software vendors with experience in the Web services market are working to provide links to the mainframe, either in their products or through tie-ins with vendors that have mainframe experience; meanwhile, hardware/software vendors such as IBM and Unisys are promoting Web services on the mainframe.

However, implementing Web service access to back-end applications on mainframes is not a simple task. A discriminating eye is required, as not all back-end applications benefit from being opened up. The key is to first select the applications that can benefit from being given front-end Web services access, and to then cherry-pick those that will give the most bang for the buck.

There is 'a lot of interest in Web service-enabling mainframe applications, particularly in Service-Oriented Architecture,' said Eric Newcomer, chief technology officer at Iona Technologies, whose U.S. offices are in Waltham, Mass. One reason is that companies want to use Web services for enterprise application integration (EAI) applications, Newcomer said. Advances in technology also help. 'Working with XML and Web services seems to be a lot easier and faster than previous solutions and it’s cheaper now, so we’re seeing more interest in off-host access mechanisms offered by Web services,' Newcomer said. 'Most of the tools out there for Web service enablement provide help through graphical wizards and GUIs like [Iona’s] Artix does so you can import Cobol and create WSDL that can easily be accessed by any Java or .NET program.'

Mike Connor, product manager for WebSphere Studio Enterprise Developer at IBM, agrees with Newcomer’s observations. “Two years ago, if you got a customer call on these things you’d have to deal with a technical committee; a year ago you were dealing with the architect level; I’d say in the last three to six months we’re seeing Web services being dealt with at the CIO level -- we’re dealing with the CIO and a few VPs and senior-level architects,” Connor said. “Web services are going to be part of any discussion at any level as it relates back to the business today.”

The interest in Web services access to the back end comes from organizations across the board. David Clarke, senior vice president of strategy and business development at pure-play Web services player Cape Clear Inc., Waltham, Mass., said his firm’s clients include “seven financial services, some telephone companies, and some Federal and state government clients in the U.S.” Clarke said government clients are interesting “because they have a lot of mainframe stuff deployed and have shrinking budgets, so they need to do integration cheaply. They also need to track information more securely because they are under increasing legislative and regulatory burdens.”

The reason for the interest is that many corporations want to use Web services “to open up functionality in a way that allows them to be accessed through Unix and other platforms sitting in front of the mainframe,” explained Jim Bole, vice president, professional services at Infravio Inc., Cupertino, Calif. For applications sitting on the mainframe, people “have to spend a lot of their time on exposing individual transactions -- CICS or IMS, or functions within the mainframe rather than entire apps,” Bole said. The advantage Web services offers over direct Web-to-mainframe integration -- as was done in the past few years -- is that Web services “gives you the ability to access not only data but application processing or functionality,” he said.

The players
IBM and a slew of existing host integration vendors such as ClientSoft Inc., Miami, HostBridge Technology, Stillwater, Okla., and InterAccess Corp. are offering Web services capabilities for mainframes. Add to that list Attachmate, BluePhoenix, Fujitsu, Jacada, Micro Focus, Neon Systems, Relativity, Seagull Software Systems, Seec, WRQ and others. Meanwhile, vendors coming to this market from the Web services side are still coy. “We haven’t seen much in the way of pure-play Web services vendors competing where mainframe services are involved, but that will happen over time,” said Cape Clear’s Clarke.

Whether they come from the mainframe or Web services camp, vendors’ solutions tend to range from simple “screen scraping” services to deeper integrations that require business rule extraction, code restructuring and the like.

IBM, the 500-pound gorilla in the field, has undeniably been making its presence felt. Cobol now has Web services built into it, and IBM offers tools that can correlate Web services into flows and generate runtime access wrapped into those services as part of the application flows; it is also building the ability to monitor applications “across the various hops that Web services entails” into its monitoring and delivery technologies, said the firm’s Connor. IBM has also added Web services support to Cobol and PL/1. With CICS and IMS, which “provide very strong transactional capabilities,” IBM has “added the capability in CICS and IMS runtimes themselves to provide both inbound and outbound support for Web services in CICS 2.2 and 2.3,” he said. This is available in a packaged component, SOAP for CICS, which is offered free to CICS v2 users. IMS will have that capability in general availability “later this year,” Connor said.

In addition, IBM has released WebSphere 5.1 Application Server, which it offers as its platform to support e-business on demand. IBM sees e-business as the ability to have applications and computer platforms available on demand through the use of middleware and Web services.

Unisys Corp., Blue Bell, Pa., offers the Dorado family of ClearPath mainframes that come with Web services-ready middleware. Built on the same Cellular MultiProcessing server architecture as Unisys ES7000 servers, they include partitions with Intel processors; integrated control over OS2200, Windows and Unix operating systems; and Sentinel self-healing, self-management technologies. Other features include the ability to shift performance among independent environments or applications on the fly, and the ability to provide capacity on demand.

Longtime integration and infrastructure software provider Iona Technologies has added two new packages that provide Web services for the mainframe to its Artix family of service-oriented integration products. Artix Mainframe Transformer and Artix Mainframe Developer run on the mainframe where they use C++; off-host, they support “anything -- there’s a lot of platforms out there, so we support .NET and Java clients, everything, through Web services and all sorts of corporate clients natively,” Iona’s Newcomer said. Artix Mainframe Transformer is used to transparently create a Web services interface for an application that has a callable entry point written using PL/1 or Cobol, he said. It takes the data types from PL/1 and Cobol, and generates the XML, WSDL and SOAP messages automatically, he added. Artix Mainframe Developer allows users to develop a new service on a mainframe “so they can access programs that do not have a callable interface, such as a batch program or a Cobol program that just runs standalone or even a standalone C++ program you need to launch through a command line rather than [through] an ATI call,” Newcomer said. Iona supports SOAP 1.1 and WSDL 1.1, and conforms to the WSI basic profile, he noted.

Cape Clear’s Business Integration Suite of products is built from the ground up using Web services, allowing users to integrate existing internal systems and then extend these to connect with their suppliers and customers. Cape Clear’s products integrate MQSeries and the messaging bus, and the firm is partnering with mainframe integration tools provider ClientSoft to WSDL-enable some of ClientSoft’s existing tools.

“Mainframe and host integration [is] a specialist area and we’re Web service specialists; we don’t want to attain deep mainframe expertise ourselves, but would rather do that through partnering,” said Cape Clear’s Clarke. “All you need is correct mapping from WSDL into the mainframe data structures and that just makes use of the WSDL standard; the core standards are well battened down and there isn’t really any particular extra effort required,” he said. “So when we work with ClientSoft, for example, they generate a WSDL interface from common data structures and we consume them directly into our platform.”

The Business Integration Suite, based on XML and WSDL, consists of four products: Cape Clear Studio, which provides an intuitive, graphical environment for designing, developing and deploying business services; Cape Clear Server, which provides a full-featured platform to support the deployment and integration of large-scale business services; Cape Clear Manager, which provides a full-featured management console for managing and adding security to users’ business services software; and Cape Clear Data Interchange, which provides a visual environment for transforming diverse data sources, such as text files, spreadsheets and ZIP files, into XML schema. Cape Clear Data Interchange also offers a runtime capability that securely routes transformed data to the appropriate back-end application.

ClientSoft, which is partnering with Cape Clear, provides secure, high-performance legacy integration solutions for .NET, Java and Web services environments. Its products, ClientBuilder and ServiceBuilder, serve as development platforms and runtime environments. ClientBuilder provides both thick- and thin-client options, and is designed for heavy workflow reengineering of legacy applications. ServiceBuilder allows enterprise developers to wrap existing legacy programs and transactions into ready-to-use .NET, Java or Cobol components or into Web services. It runs natively on the mainframe, and the company claims it is the only product on the market with bi-directional Web services support -- it can publish legacy transactions and allows legacy applications to consume Web services.

Yet another vendor is OpenConnect Systems Inc., Dallas, which offers the Mainframe2Web Secure Solutions Series of scalable solutions to Web-enable host applications. This family of products offers secure access to business information and apps on mainframe, midrange and Unix systems. It consists of Secure Access Solutions --WebConnect and WebConnect SSO; development tools -- eXtremeVista and Visual 3270; and Web service solutions -- xmlConnect.

Where there are services, management is required, and vendors are moving into the Web services management area. Infravio offers the Ensemble Web Services Management Suite, which lets users manage their Web services. Users can manage the delivery of a service to a particular consumer or consuming application, said the firm’s Bole. “For example, I might decide that certain business processes have priority over others and, based on a services management environment, you can scale up or down the available bandwidth or processing power on a per-consumer basis. That’s important because once back-end services become available, people will begin to use them widely,” Bole said. Infravio Ensemble’s features include version control, change control, multiple version management, and performance and usage tracking. Ensemble Distributed Broker runs on most J2EE platforms, including BEA WebLogic Server, IBM WebSphere and JBoss, as well as HP NonStop. It will soon be available on .NET.

Toward the data center
John Fowler, Sun’s chief technology officer, predicts that the data center’s role will shift from one of managing boxes to managing services. He outlined three stages of deployment of Web services in the data center. The first stage is using Web services as management interfaces; stage two is the establishment of standardized systems for describing the data center, its services and infrastructure; the third stage will be the ability to change the data center on an ad hoc basis to suit current business requirements.

IBM’s Connor characterized the stages as follows: In the first stage, companies take a tactical view toward Web services, linking existing applications through some sort of interface; in the second stage, companies move toward more of a strategic view of Web services, “perhaps implementing not only access to messaging, but also an entire Web services infrastructure,” said Connor. This infrastructure could be based on the Business Process Execution Language for Web Services (BPEL4WS), which, supported by IBM, Microsoft and BEA, has beaten out BPML in the war over business process management software standards. BPMN, the Business Process Modeling Notation of BPML, may be layered on top of BPEL4WS, giving users a single unified stack of standards for Web services choreography. The third stage, Connor said, would see Web services becoming platform- and application-independent.

“In other words, everything will be a Web service and maybe we’ll have a true Web services environment,” he added.

Larry England, IBM’s lead architect for WebSphere Studio Enterprise Developer, said that developments are occurring in each of the three phases. “The idea of Web services is a way to manage interfaces into applications or capabilities on back ends and to have those things be well defined, not only in terms of their inputs and outputs, but where they can be deployed, what their scalability is and whether they can or cannot be run in parallel . All the characteristics of those services need to be identified and managed,” England said. These are used as a base construct to lead into the second phase -- the ability to use those components as building blocks for standardized systems, he noted. That “gives you the capability to be very flexible in how you adapt to the marketplace, and lets you reuse and re-purpose the components for new types of applications,” England said. The third phase is the ability to move into an on-demand world. “All of these things build up toward that capability of being in an on-demand world: The ability to integrate the componentization, if you will, to use these well-defined services and have them deployed in a grid fashion, [as well as] the ability to dynamically shift the workload to another processor or subsystem. [In addition, to] have that all work with the front end user as demand goes up and to then bring online more compute power that can host these well-defined services,” England said. “I see those things evolving, but I see some of phase three today.”

Infravio’s Bole said corporations are already in the first of Fowler’s three phases. “It’s about having ubiquitous access to all the management information you need to monitor and manage services so that Web services can be managed like any other enterprise asset,” he said. Infravio’s Ensemble Web Services Management Suite “leads you down to phases two and three of Fowler’s rollout,” Bole said. By the time a corporation gets to phase three, it could set up quality of service models and service level agreements that would allow the enterprise infrastructure to be automatically based on load and need.

“The mainframe will be a provider of highly scalable services, and that’s where it has a huge role -- in Web services parlance, it will be called a superprovider,” Bole said.

Technical issues
Web service-enabling mainframe apps can be a tad complicated, and IT managers need to develop a systematic approach to the task. Over and above the usual practices, you should pick small, highly visible, easily completed projects that provide major brownie point value and then break them down into easily manageable tasks.

Broadly speaking, there are three ways to Web service-enable mainframe applications: Provide native Web services on the mainframe, integrate the VM messaging bus or directly Web service-enable the mainframe, Cape Clear’s Clarke said. Native Web services on the mainframe are “in the ‘technically possible but not likely any time soon’ class,” he noted. Integrating the VM messaging bus is done through the MQSeries messaging bus, Clarke added. However, in some very sensitive applications, or where the messaging bus or MQSeries is not implemented, IT will have to directly Web service-enable the apps by building WSDL representations for some of the mainframe constructs, WSDL generators for COM area descriptions, and Web service representations for some of the key mainframe, data and app elements, explained Clarke.

“That’s quite a specialist activity, and what we’re seeing is that companies with existing mainframe integration experience will look to extend their products by WSDL-enabling them,” he said.

The first thing to do is to select the correct mainframe application to Web service-enable. “Some applications are appropriate for Web services and some are not,” Iona’s Newcomer said. “You must decide whether it’s worth the investment to Web service-enable the mainframe application first.” Whether or not the investment is worth it depends partly on the amount of work required on the mainframe to create access through Web services, and partly on whether the data will be shared by other applications. In terms of mainframe applications, IMS is easier to work with than CICS or other mainframe applications that are more synchronous or callable, because “for IMS you put a Web services shell around the IMS queue and that’s pretty straightforward,” Newcomer said.

Once the app has been selected, you must decide what data needs to be shared with other mainframe applications and define an XML schema for it. This could be difficult, but “you need to focus on it because you might just need a subset of the data to be shared, not all the data,” Newcomer said.

After the data format is ready, you can then construct the service description and WSDL, and mainframe vendors offer several tools to help. Then, when the Web services description is created, you can use the tools to deploy the service and put it into production, Newcomer said.

But performance levels are an issue. “Typically, mainframe systems require a very high level of availability, security and performance, and the risk is that if people start randomly Web service-enabling mainframe environments, [they] will have expectations of mainframe levels of quality of service [QoS], but that QoS may be compromised,” said Cape Clear’s Clarke. The solution: IT should pay attention to proper design and integration, and “make sure that the infrastructure you use -- the Web service enabler and host service -- can offer robust quality and service so there isn’t a weak link. There’s no magic bullet,” he noted.

Web-service enablement is most useful in environments that are “not the high-volume, high-transaction rate types you’d see in a typical EAI kind of application, but the next tier down in the enterprise, where people are looking at extracting data from mainframes and publishing in portals,” said Infravio’s Bole. Even then, IT must be selective. “The trick is to define very coarse-grained services that have relatively small bandwidth requirements but carry a lot of information and provide a lot of functionality at the back end, so you don’t have a lot of network chatter going on,” he said. A good example would be the ability to post an order on a portal that does a lot of supply-chain integration on the back end, Bole said. “You want high-value services that are highly manageable, valuable, monitored and secured.”

Next, pay attention to the service architecture. “There’s an incremental way to implement Web services architecture,” Bole said. “You can do XML transformation piecemeal rather than system-wide.” At the same time, IT must make sure it is delivering objects that will be reusable over time.

Despite all this, the technology side is relatively simple; people issues will be harder to resolve. Inevitably, there will be a clash of cultures between the mainframe and Web services camps, said Cape Clear’s Clarke. “The technology issues are relatively addressed, but the soft people issues are more interesting.” Those words will resound with experienced IT hands who lived through the culture clashes between data center types and new technology advocates during the introduction of the PC, and client/server and distributed computing to enterprise systems.

Please see the following related stories: “Relativity charges ahead in legacy transform world” by Jack Vaughan

“User story: Raytheon on track” by Jack Vaughan