In-Depth
It’s the mainframe’s turn to get the services treatment
- By Deborah Melewski
- October 30, 2003
 While the software industry focused on Java, .NET and Web services in recent years, the mainframe became an afterthought. It was expected to go away.
But IT shops are now waking up to find that the mainframe still runs the key enterprise applications, and that the mainframe room is due for some “home improvement.” |
Application development managers are discovering that, when it comes to refashioning mainframe apps as Web services apps, there are many different ways to go. And as Gartner Inc.’s Dale Vecchio said, they may open up a Pandora’s box if the dig too deeply into that legacy code. Developers need to let more -- not fewer -- users into the mainframe world, and some will opt for host-to-Web, GUI extensions and screen-scrapers as the best means to start to bring the mainframe into the realm of Web services.
When the Minnesota Dept. of Public Safety Driver and Vehicle Services (DVS) Division in St. Paul decided to integrate its legacy mainframe applications with Web services, the division chose to go with the Verastream Host Integrator product from WRQ Inc., Seattle.
According to Judith Franklin, manager of enterprise technology support at DVS, the division’s goal was to improve data quality and accuracy, as well as turnaround time to business partners and the drivers of Minnesota. Challenges were plentiful. To begin with, the system in use was a largely paper-driven environment, based on a mainframe that was more than 25 years old.
When Franklin was brought in, processing a motor vehicle title transfer took between three and six months. It was not uncommon for the processing of a driver’s license to take up to six weeks. Most of the delay, she said, was caused by the amount of time it took for the paperwork to arrive at the DVS and be entered. In addition, there were a lot of data-entry errors.
“Based on our business model, if you needed to get a driver’s license or renewal, or register a vehicle or transfer a title, you had to go to our business partners who could be the cities, counties or private individuals in business for themselves,” said Franklin. “You would go there, do your paperwork and they in turn would send us the paperwork. And there it would sit, a great pile of paperwork that would eventually trickle to the top where our data-entry people would then enter it.”
The DVS system was running on an IBM S390 mainframe with Cincom’s proprietary Supra database, which was proving very expensive to maintain. “We needed to do something dramatically different,” Franklin said.
Franklin found two or three Cobol programmers who knew the system inside and out. As is often the case with legacy systems, there was very little documentation. The screens were typical CICS screens, and could not be released to business partners or to the public because they contained too much information and were difficult to read.
Franklin’s first goal was to do something effective -- and in a relatively short timeframe -- that would allow the DVS to start publishing on the Web and provide a service-oriented approach for doing transactions and real-time data entry with business partners. It was not necessary to enhance the current system because the DVS planned to eventually move away from that database.
“We required a tool with a minimal learning curve for Visual Basic programmers already on staff, rather than having to bring in Java programmers and build up expertise,” said Franklin. “We also wanted a tool flexible enough to fit into wherever we would be doing our core application development as we grew.”
Using Verastream Host Integrator with Microsoft Visual Studio, and running on Microsoft IIS with ASP, the DVS went live with two Web sites for driver’s license applications in less than two months. One site was for the public, while the other was a private authenticated site for business partners.
The DVS built Web apps that go to the Verastream middleware, which then goes to the mainframe and returns data with records that are deployed and formatted for readability. The DVS is also merging its Verastream model with a variety of SQL databases.
A site for law enforcement was then added. “This has been a huge turnaround for the state,” said Franklin. “Our law-enforcement partners say that the services we now provide are the best tools they’ve been given since the mobile radio.”
Prior to Web services, law-enforcement officials had to spread out pages of papers; with the new system, they can access a current, readable driver’s record. More than 10 million photos can be stored using the SQL database, and the records are integrated with the photos.
The division has taken data entry out to all deputies and agents in the state, and has been doing real-time transactions for about a year. “This has allowed us to withstand budget issues that all states are going through, and it has increased the accuracy of our data,” said Franklin. “The DVS has saved some $260,000 in programming/legacy storage costs, and has cut license processing times by 75%.”
But the change was not completely smooth, admitted Franklin. The DVS went from a batch-processing concept to one that suddenly opened up 24x7. There were also bandwidth problems when the DVS started getting more than 27,000 hits a day.
“In many ways, our mainframe people weren’t prepared for the increase in production, CPU cycles and database calls, which more than doubled in the first three months after opening up our mainframe to the Web,” she said. “And we learned that it would all double again during the next three months.”
Also a concern: maintenance. DVS people brought everything down for maintenance one Saturday morning. “It was a one-time job on a Saturday morning, and suddenly the motor vehicle site is down and my beeper is going off,” said Franklin, admitting to some shortsightedness. “The concept is so different now that you might unwittingly let things like that slip by.”
Performance is key
“People are finding that they need to move their applications to the Web to make it easier for their customers and partners to do business with them,” said Scott Rosenbloom, chief strategist at WRQ. “One of the challenges is that some 60%–70% of their information and business logic is stored on legacy systems.”
Companies need to move quickly to the Web, so re-creating business logic and data on new platforms is just not possible, he added. “They’re starting to step back and say, ‘We already have that business logic and information. It’s reliable, and it’s been on the host maybe 15 or more years. Let’s find a way to expose that to the Web to make the creation of these new applications quicker and give them higher functionality.’”
WRQ’s Verastream product is a runtime server with design tools. It works with products like Microsoft Visual Studio .NET, BEA Workshop and IBM’s WebSphere. “We expose the information with Verastream, and that is used to build the actual application,” said Rosenbloom.
The WRQ tools also support setting layers of granularity. Many approaches to integration in the past failed, noted Rosenbloom, because when people started using services like CORBA they did not change the way they viewed the calls, even though they were going over a network where there is a lot more overhead and latency.
“In Web services, you can have this same challenge -- if you just expose the existing API via Web services, you could have very low-level, fine-grain calls going on. For example, if you’re updating a field or a record as opposed to sending an entire document full of many records that you may want to submit all at once,” Rosenbloom explained.
“It’s very important that you implement Web services with the right level of granularity when you make this Web service call because there is a lot of overhead associated with this call,” he added. “You’re creating an XML document. You’re shipping it over a protocol like HTTP over the network, and there are a lot of different things going on; you want to ensure that when you make that call over the network, it will do a lot of work for you before it returns.”
Like WRQ, Bellevue, Wash.-based Attachmate Corp. has also been in the enterprise information access business for 20 years. The firm’s Web services options for integration with legacy systems include myExtra!Smart Connectors, with access to IBM 3270/S390 or ZSeries mainframes, IBM5250 midrange apps or Unisys mainframe applications.
myExtra!Smart Connectors includes a GUI Design Studio for creating logic to interact with the back-end system; a runtime component that interacts with the back-end systems and communicates with the newly written front-end app; and a Management and Control Services component for administration of security, performance and deployment from a single point.
“One of the biggest problems our customers face is that they have huge installations of mainframes and huge installed bases of users requiring access to those systems,” said Markus Nitschke, Attachmate’s vice president of marketing. “After Web services are developed, there needs to be a way to secure them, authorize all these users, and ensure failover and clustering.”
Stability and performance were driving factors in developing the runtime environment, he said. “The Connectors run on the mainframes; they also support and run on any type of Unix, Linux and Windows platforms.
“Instead of introducing another tier for integration, [some customers] asked us to provide Web services as close to the source, being the mainframe application or database, as possible,” added Nitschke. “Those customers have turned their mainframe into a server that can directly participate in a Service-Oriented Architecture.”
Why mainframe?
Mainframe computing has gone through much iteration during the last 30 years. The mainframe has been eulogized, looked to for distributed processing and client/server computing, re-architected, re-invented, screen-scraped, GUIed and dot-commed, and its applications have been revamped for Y2K. It has also been proclaimed dead many times. But that was only a rumor, as Paul McCartney once responded to the proclamation of his own demise.
The old 3270 terminals and green screens have all but gone away, however. Many end users, particularly those in a Service-Oriented Architecture, probably cannot tell you where their mainframe resides or even whether they use one. But the apps designed for those terminals are still running; in corporate IT, most agree that the mainframe is well recognized for its many strengths and will not be going down any time soon.
“Nobody argues about the value of legacy applications,” said Attachmate’s Nitschke. “Most of our customers have realized that the whole push of turning the mainframe off is not only not cost-effective, but it would not work out to their advantage because all of their business processes, business logic and all the values they have in terms of internal processes, reside on these legacy systems.
“The issue is enabling those legacy systems to actively participate in the Service-Oriented Architecture,” he added. “The bad news is not the system itself, but that it was never meant to participate in this new architecture.”
There is a lot of value and experience in those older, much-customized systems, Nitschke said. “We just need to make use of those systems in a more efficient and cost-effective manner. We need to make these older technologies participate in new projects with Web services.”
Nitschke noted that several customers have achieved cost savings by using a centralized strategy. “The pendulum is now swinging away from a very federated, distributed IT system to a more central management and control of IT projects,” he said. “And it turns out that this central tool in a lot of cases is the mainframe itself.”
“It’s always been important to integrate with the mainframe,” said Adrian Gosbell, Uniface product manager at Compuware Corp. in Amsterdam, The Netherlands. “But the whole uptake or trend has been a bit slow. I think that could partly be because there hasn’t been the same technology available on the OS/390 platform as there has been on other platforms. Integration has been possible on platforms using COM and DCOM, [and] it’s been available on Unix, Java and other platforms using CORBA.”
Gosbell said that the only way to get integration working on the mainframe has generally been to write to a very low level. “Web services solves that problem for the mainframe because Web services speaks the common language of XML,” he noted.
Legacy renewal can be on various platforms, said Gosbell. “For example, there may be a need to deploy a common application across the entire infrastructure. There may be a need to integrate with existing apps through CICS and also with direct connections to Cobol and even to applications written in Assembler. There may be applications running Adabas or Natural that a company may want to keep running.
“There is no reason to replace an existing, fully-functional application with something else that does the same thing,” said Gosbell. “Why fix it if it isn’t broke? You can tightly integrate with XML and then plug into a Uniface app. We can also have some Java apps deployed on the mainframe, and those applications can execute Uniface services as a Web service.”
After speaking with representatives of its user community in Germany, said Gosbell, Uniface has discovered an uptake of Linux in Europe. “Linux is being pushed quite hard on the mainframe, and using Web services provides the ability to move between various partitions on the mainframe,” he said. “Some of those partitions could also be Linux. This is a way of actually executing integration between those partitions.”
Integration challenges
“The biggest challenge we see is that people need to get to market quickly,” said Gosbell. “But also very important, and oftentimes forgotten, is the ability to implement change. Uniface 8.3 has a model-driven architecture, so the ability to implement change in an application means that people can think for the future without finding hidden costs.”
“Customers are very attuned to security and administration challenges,” noted Attachmate’s Nitschke. “In addition, large organizations have certain performance requirements. They know they can’t introduce a bottleneck that will cause performance problems down the road. Components have to be written in a way [so that they can] handle very high transaction loads and basically are not the weak link in that chain of applications.”
Another area requiring attention is the knowledge level or skill sets needed to build these applications. “We’re talking about legacy systems, highly proprietary interfaces, and Cobol that has grown for years and years,” said Nitschke. “On the other hand, you may have very young Java developers that have been given the task of interacting with these systems.”
With a GUI design tool, Java developers do not require knowledge of Cobol. “Up front, they do have to understand the application and the logic that resides on the mainframe, but they don’t have to understand the underlying technology,” he explained.
Why not XML?
Some corporations, as they ponder mainframe renewal, will look for deeper integration of mainframe and non-mainframe data. Some will turn to deeper XML integration, because XML is the foundation for modern Web services. XAWare, Colorado Springs, Colo., offers XA-Suite for XML-based enterprise information exchange and data integration, and its software can play in the mainframe arena.
“Many of our customers have mission-critical applications that solve a critical business process, and they need to share that information so they need to get it into XML,” said Rick Cloutier, vice president of marketing at XAware. “To do that unobtrusively is very important not only from a technology standpoint, but from a political standpoint. You don’t want to turn over the apple cart.”
XAware has deals in place to furnish connectors between its J2EE-based integration components and a variety of legacy and mainframe environments.
XA-Suite creates XML views from any type of data, including legacy, enterprise application or relational data. The data is presented as a single, logical document. At design time, the customer can specify what data sources they need to go against, along with what XML format they want to convert the data to or from.
According to Kirstan Vandersluis, XAware founder and CTO, accessing data, particularly on the mainframe, can be a challenge. “If a company’s legacy system is completely isolated, if the only thing they’re running is a 3270 application and they haven’t integrated it with the rest of their enterprise, we bring in an adapter to access their legacy data source directly. Then, obviously, we would need to make sense of that, so there are also data analysis challenges,” he said.
“We expose the data, but somebody has to have an understanding of what it means, especially for a database created some 30 years ago,” explained Vandersluis. “People created fields and did their best to name [them] according to their purpose. But normal programming practice back then was to compress a field name to six or eight characters. Somebody has to figure out what each data field means, and what its relationship is to other data within the enterprise. There are varying degrees of documentation available. There are also a lot of business rules buried in Cobol programs, and they can be difficult to uncover and replicate.”
More often, Vandersluis finds that companies have gradually created interfaces to their legacy applications. They may have a messaging interface on the legacy system that can handle requests from other applications. A company that adopted client/server technology may have already solved that piece of the equation.
Another way to access data is screen-scraping, which Vandersluis said is not the best way to go because of performance problems. However, he added that one benefit of screen-scraping is that it does preserve the existing business rules already coded into that mainframe application.
ROI concerns
Before development even starts at most companies, there are certain requirements a vendor has to meet. A hot topic is cost-efficiency not only in terms of product licensing costs, but what it will cost to administer and maintain the solution after it is deployed.
“Our customers are explicitly asking for ROI numbers before the project is even launched,” said Attachmate’s Nitschke. “The numbers we hear quite often are that the rollout may need to be completed within three to four months, and that payback has to happen within eight to 12 months. Every integration project that we encounter has to achieve ROI in less than 10 months.
“This is the first time we’re finding our customers explicitly asking for that type of turnaround,” explained Nitschke, adding that the importance may also be due in part to the economy today.
“Customers are looking for very efficient solutions, particularly in the integration space,” he said. “They’re looking for solutions that have an immediate impact on the bottom line.”
People prefer projects to be done in three to six months now, as opposed to 18 months to two years, WRQ’s Rosenbloom indicated.
“A service-oriented integration approach allows you to reuse existing assets within your organization,” said Rosenbloom, “and Web services is a great way to expose those services.”
Please see the related story "My meeting with a legacy transformation architect” by Deborah Melewski, or click here to go to special Web-only material on mainframes.