In-Depth

Manage Java apps for premium performance

• By Peter Bochner
• December 31, 2002

Companies are turning to Java applications not only because they deliver benefits in terms of performance and scalability, but because they also link applications and services over multiple platforms and generations of technology, from the creakiest legacy systems to the shiniest new Web services. And because Java applications are closely tied to business processes, they can help maximize their efficiency.

But Java applications are complicated. As Valerie O'Connell, managing director of enterprise systems management for Boston research firm Aberdeen Group, put it, they ''have a truckload of moving parts.'' To see how a Java application is behaving, an administrator must understand how all these moving parts fit together.

And if a Java application is behaving badly, there are so many possible causes -- the response time of an EJB, the availability of Java database connections, a dawdling servlet, a JVM leaking memory or even the application infrastructure itself -- that customers are ''no longer satisfied with a simple availability-type metric,'' said Jim Byrd, a senior product manager with BMC Corp., an enterprise management software company based in Houston. ''They need a comprehensive understanding of what's happening inside the application and its components.''

Unfortunately, Java applications often do behave badly. According to DD Ganguly, a technology strategist in Computer Associates' Office of the CTO, the inclusion of J2EE application servers in enterprise application architectures took one set of complexities out of the picture -- messaging, clustering and database connectivity -- but introduced ''a new set of technologies that we need to learn and manage.'' The very complexity of these Java applications is responsible for stressing their performance at a time when application performance is more critical than ever.

As a result, traditional ''best practices don't cover a lot of the newer situations out there'' related to Java, said Thomas Murphy, senior program director for application delivery strategies at Meta Group in Stamford, Conn.

A new generation of management tools provides the visibility into Java applications needed to diagnose a performance problem, and offers up possible solutions. Unlike traditional test and load balancing solutions, these products can diagnose application problems in the production phase as well as the QA phase. But the makers of these products prescribe doing proactive management before the application is rolled out. Once Java apps are scaled out to large numbers of users, figuring out what is causing a problem is a ''major bitch,'' according to one observer.

That is because few of the tiers inside a J2EE application server environment -- the main server, the operating system, the Web servers, the application server and the database servers that feed data to the application server -- are likely to be instrumented for performance measurement. Then there is the enigmatic Java virtual machine itself, which allows Java applications to run natively on any machine. ''JVM is like a black box,'' said Tom Mulvehill, senior product marketing manager at Precise Software Solutions, Westwood, Mass. ''It's hard to understand what's going on in there.''

''The big problem is being able to understand where the problem is,'' said Phil Buckellew, product line manager for transaction performance in the Performance and Availability Group at IBM Tivoli.

And the more tiers, the more time it takes to resolve a performance problem. ''In a large company, it takes a lot of pull and tug to make the changes of uncovering and resolving problems,'' said Kevin Gallagher, vice president of research and reporting at Newport Group Inc., a research firm in South Yarmouth, Mass. Newport is about to release a study that puts the average time of resolving a performance problem, from the time the trouble ticket is pulled, at 25.8 hours.

Providing the right level of management information without degrading the performance of the system the app is running on is like walking a tightrope. The challenge for management vendors is to strike a balance between collecting useful information and not imparting a lot of overhead on the system the app is running on. According to Precise's Mulvehill, using probes, debugging tools and load balancing tools affects the application whose performance you are trying to monitor.

The trick is to collect the detailed information you need to monitor the performance of your app in a way that does not affect overhead. One solution, said Meta Group's Murphy, would be for the tools to let the customer do a rough analysis; if the customer then sees a problem, instrument everything below that level.

At Fannie Mae
Firms cannot afford to wait for consumers to report performance problems. That would be fatal for Nexstar Financial Corp., Creve Coeur, Mo., which, like most mortgage lenders, relies on online technology to speed up the approval process. When a customer applies for a mortgage online, Nexstar processes the information and sends it via a gateway to computers at Fannie Mae, a source of financing for home mortgages, which then sends back the findings needed to approve the mortgage.

Once mortgage lenders reach a certain size, they need more functionality than packaged loan applications can provide, and so develop their own software to communicate with the sites of underwriting engines such as Fannie Mae. Three years ago, Nexstar contracted with EoTek, an Evergreen, Colo., Java development house serving the mortgage industry, to develop a Java application that could bridge Nexstar's online forms applications and data formats with the back-end mortgage approval system operated by Fannie Mae.

The gateway's service levels were critical, but performance was slowed because the Fannie Mae site employed a proprietary protocol. To determine how many mortgage applications it could send through to Fannie Mae, and how fast it could do so, Nexstar needed more visibility into how the distributed components of its Java application were performing.

Because of all their moving parts, Java application performance usually needs to be managed in real-time. ''In some industries you may not need to worry about every second's worth of performance, but in a Web enterprise, if you don't manage in real-time, you're going to miss a lot of changes,'' said Bob Ure, director of marketing for Altaworks, a Nashua, N.H., developer of performance management software for Web enterprises.

Nexstar needed to add instrumentation to its app, ''so that we could know how the gateway was performing and whether it was experiencing difficulties, such as communication circuits,'' said Mark Carlson, Nexstar's VP of technology. And, he noted, Nexstar ''needed to know about all these things in real-time.''

Carlson had several options. Nexstar's IT department could attain some of that information by running queries -- ''analyzing log files after the fact'' is how he described it -- but it would not have been in real-time, nor would it have provided visibility into the application components. Another option was to use SNMP, the most often-used method for adding management to Java applications, to create custom Management Information Base (MIB) extensions to look into application data. But even though SNMP requires little code to implement, ''we wanted to standardize on something current, rather than adding an older MIB that we would hand-code ourselves,'' said Carlson.

A third possibility was to use the Application Response Measurement (ARM) API. ARM is a spec proposed by Hewlett-Packard and IBM that can be used by developers who want to collect performance information by instrumenting the business logic of their apps. It does this by embedding function calls that can be captured by an agent in the software. However, the technique is still quite complex and requires too much hand-coding for most developers' taste.

The Nexstar infrastructure includes an Oracle DBMS, IBM WebSphere application servers, Tomcat open-source Web servers (''We develop under Tomcat and deploy under WebSphere,'' said Carlson), and webMethods application integration software. It uses packaged applications for its enterprise functions, such as Siebel for CRM and PeopleSoft for financials, but typically writes custom Java applications for its mortgage business. These include a decision support app for routing loans to different credit providers, and one that configures pricing for different lenders.

Enter JMX
Because several products he was using had already implemented the Java Management Extensions (JMX) management spec, Carlson wanted to explore that avenue further. JMX is an API for the instrumentation and management of J2EE applications. Currently an option, it will become a mandated standard when J2EE Version 1.4 is released in 2003. Its popularity is growing: In 2001, only 10% of J2EE applications were JMX-enabled, according to Gartner Inc., a Stamford, Conn.-based research firm; they expect that number to reach 50% in 2003.

JMX makes it easier for customers to get raw management information data and turn it into data that gives them a view of their application performance. Because it uses familiar Java semantics, it will make it easier for developers to build management into apps. ''Our customers want to plug in management and instrumented solutions around standard APIs,'' said George Kassabgi, vice president of engineering at BEA's Foundation Division.

Open-source JMX solutions did not have everything Nexstar needed in terms of instrumentation. Further, customizing JMX to Nexstar's situation might have meant rewriting the application, and Nexstar was ''looking for a way to expose attributes of our application without having to rewrite it,'' said the firm's Carlson.

Carlson heard that an upcoming version of a software framework from AdventNet Inc., a provider of J2EE management software in Pleasanton, Calif., would bundle a pre-written implementation of JMX. After reviewing a pre-release version, Carlson joined the beta program. AdventNet allowed his department to do initial setups without rewriting the underlying application code from scratch to support the JMX extensions. Using the AdventNet GUI, they simply chose which attributes they wanted to expose. All that was needed was a couple of lines of code.

By automating instrumentation, AdventNet's ManageEngine framework allows developers to customize the management of their apps to meet business needs. ''Most developers don't want to deal with instrumentation until after the application has been deployed,'' said Giri Giridharan, vice president of product marketing at AdventNet. ''They'd rather spend time developing applications than reading about manageability.''

Or as BMC's Byrd noted, ''The vast majority of customers are not instrumenting their application right now.''

In the past, the process of exposing application attributes, like response time, or the functions to be managed required a big investment in time, effort and knowledge. ManageEngine allows developers to graphically pick the attributes they want to manage by pointing at Java Archive files or Java class files and representing them as EJB components. AdventNet's approach does not require developers to create custom MIBs or use MIB editors; it automates the creation of the management beans, or Mbeans, JMX needs to manage an app.

''We were able to expose the information we wanted, even read-only information,'' said Carlson. The developer who configured the JMX features on AdventNet had no JMX experience, but after two days learning the JMX Studio portion of ManageEngine, he was able to expose that information.

Altaworks' Ure said that in 2003, users can expect ISVs to offer performance management as an add-on service to their packages, with JMX functionality built into applications to decrease support costs. They will be able to turn it on from their office, run it over the Web and check out the performance of their software running in the customer's environment.

''Some customers may balk at having a performance management agent that wakes up when the packaged software is installed, if it's accessible over the Web by a third party; but with VPN or encryption technology, that will be less of a problem, and customers will see benefits of having that proactive management,'' he said. ''It could lead to packaged software vendors viewing things proactively and telling the customer, 'If you do these three things, you'll be able to process 20% more transactions.'''

See the related story, ''AdventNet pursues more manageable Java,'' by Peter Bochner.