In-Depth
Managing .NET, J2EE apps
New York City's former Mayor Ed Koch was famous for asking his constituents, ''How am I doing?'' Their answers were based on the quality of their experiences living, working and navigating in the city. Similarly, when today's business executives ask IT, ''How am I doing?,'' they do not want to know if the server is up or down, if there is enough storage space or memory, or why the network failed. They want to know the quality of the user experience: How many people ordered widgets today? How long did it take them? Did invoices go out as soon as the orders shipped? Why are online shoppers abandoning their carts?
The emergence of new infrastructure management products that target application monitoring and management -- especially for heavily network-reliant .NET applications, in which many of the services are distributed -- are helping IT organizations provide answers to questions such as these.
According to a recent report from Giga Information Group on the infrastructure performance management market, there is significant vendor activity in the application management space, including an anticipated demand to manage .NET and J2EE platforms and the addition of application quality as another metric. According to Giga, most of the $300 million in growth in the market between 2000 and 2001 went to smaller vendors offering point solutions and application-oriented monitoring technologies. Giga defines infrastructure performance management products as those that monitor, detect and identify abnormal behavior of the servers, operating systems, storage, middleware and applications, all tied together through a network.
The idea behind a user's quality of experience ''is to either directly or indirectly monitor the performance and availability of the application from the end-user perspective, so I can open the application and the performance is acceptable,'' said Mark Ehr, senior analyst at Boulder, Colo.-based Enterprise Management Associates (EMA). ''Reinforcing that is the movement toward service-level management, which guarantees users a certain level of performance. So you should have a way of monitoring that.''
Ehr said there are a number of ways to measure quality of experience, including:
* Via an agent on the end user's machine;
* Monitoring transactions as they come through on the server side;
* A hybrid approach that measures a representative transaction and requires lower overhead; and
* Bottom-up monitoring, which codes and instruments the app server.
''A lot of Web services vendors are looking at a mechanism where they can combine all [of the above measurements] with classic systems management metrics, so you can start at the end-user perspective, detect the problem and then rapidly drill down to the cause of the problem,'' said Ehr. ''In a perfect world, you need access to everything going on in every layer.''
As organizations look to add a Web services infrastructure and Web services applications to their portfolios, the challenge of application monitoring and management becomes more complex. Said Ehr, ''.NET applications are very heavily dependent on the network, more so than other applications except J2EE, because so many of the services are distributed. Failure in one part of the network could be pretty serious. And those services aren't necessarily in the same physical network -- they could be co-located at a service provider or a business partner. A lot of these services won't be under the purview of people responsible for the application, so there's more potential risk.''
''You get into issues of environments you don't have total control over, or ones you don't have monitoring privileges for,'' agreed Steve Dykstra, product manager of Compuware's Vantage systems management suite. ''Honestly, we don't have answers for all those issues; most of the activity now is internal; people are dipping their toes into Web services.''
With .NET, said Scott Matsumoto, chief technical officer at Xtremesoft, Woburn, Mass., ''you have all the basic challenges, but you're adding more complexity on top of what you have. With Web services, operations management and service-level management are coming together. The differences and relative importance of each, and to whom they're important, starts to blur. SLA [service-level agreements], which is primarily a business concern today, now becomes a concern to operations people.''
In the area of application monitoring for .NET, EMA's Ehr, in a recent report, cited several key considerations:
*Measurement of end-user quality of experience in complex .NET environments;
* Determination of the source of .NET app performance bottlenecks;
* Verification of performance and availability of third-party .NET service providers, particularly those that have provided SLAs;
* Identification, management and monitoring of .NET users; and
* Obtaining .NET visibility across all tiers of the IT infrastructure.
Ehr believes Compuware is one vendor that is well positioned in the .NET application management space. ''What Compuware did is they took their base product [Vantage] and added support for SOAP; then they added to that to look for the .NET messages coming across. Once they latch onto one, they can tag it so you know it's a .NET transaction.''
Compuware's Vantage suite provides visibility into all layers of the IT infrastructure. Specifically addressing the end-user experience is ClientVantage, which can run in either passive or active mode. ''Support for active mode allows you to easily create a script that will automate the user interface, which is nice if a business user has a problem,'' said the firm's Dykstra. ''You can play back the script and see the way you run the transaction -- if you get poor response time for something you chose to measure, or if the app didn't respond in a way you expected. It captures the information flow from the client to the first-tier server, and allows you to diagnose if the problem is on the client side or first-tier server. The nice thing about true end-user experience measurement is that you can provide reports that managers are interested in.''
Another product in the suite, ApplicationVantage, provides detailed bottleneck analyses on production .NET applications, tying in the underlying IT components that interact with them. The product captures application transactions from client and server perspectives. SOAP and XML support provide .NET-specific transaction captures, and gathered metrics enable application developers to determine which SOAP requests are taking the most time to complete.
Other application management players positioning and competing in the .NET and J2EE platform space include top guns Computer Associates, BMC Software, IBM Tivoli and Hewlett-Packard, as well as smaller vendors such as Mercury Interactive and younger players such as Xtremesoft.
Kicking the tires
Most organizations are in the ''kicking the tires stage with .NET overall,'' said EMA's Ehr. ''Most are waiting for Microsoft to release the .NET Server.'' But they are not waiting to add an application-specific focus to their infrastructure management arsenal.
At a major Midwestern retailer, a problem with a guest services application running at its stores led the company to Xtremesoft. The customer-facing application, used by in-store service desk personnel, would run fine for four to six hours, then hang up, according to the manager of the e-technologies group. ''We suspected it was a memory leak, but it was hard to nail down,'' he said. After evaluating two products to help track down the problem, the firm chose Xtremesoft's AppMetrics for Transactions for the COM+ application server.
''We initially installed AppMetrics in a lab environment and did testing of the application,'' explained the manager. ''We noticed a peculiar behavior immediately. This led to the discovery and fixing of a system-level memory leak. Once we were past that problem, we ran AppMetrics in the production environment and identified a COM+ component in the application that needed to be fixed as well.''
Since resolving that issue, the retailer continues to use AppMetrics in the lab, monitoring applications during the testing phase. ''The idea is to catch things before they go out,'' noted the manager.
The manager said Web services are not under consideration for internal applications at this point, but are on the table for partner-to-partner applications. He said Xtremesoft would have a role there as well, ''when we're doing construction of those applications and they get closer to the testing phase. We will use it as a monitoring tool in the testing cycle.''
Microsoft originally approached Xtremesoft in 1998 to provide a monitoring solution for organizations that needed to monitor the health and performance of applications built using the Microsoft Transaction Server (MTS). Today, Xtremesoft has two products: AppMetrics for Transactions and AppMetrics for BizTalk Server. Xtremesoft's software solutions integrate with a number of leading management consoles from a variety of vendors, including Microsoft Operations Manager, Computer Associates UniCenter, Tivoli's family of operations management products, Hewlett-Packard OpenView and NetIQ AppManager.
Xtremesoft CEO Michael Dexter-Smith said the company would announce its .NET direction in Q1/2003. ''Some of it is predicated on the release of .NET Server and the next version of the .NET Framework. We only focus on Microsoft apps; we're not going to go to J2EE.''
The user's point of view
PlaceWare Inc., Mountain View, Calif., delivers Web-based conferencing and collaboration services to its customers. ''I benefit greatly from the Internet and suffer from it, relying on customers' PCs and customers' networks that I have no control over,'' said Michael Ziock, senior director of hosted services for PlaceWare. ''We needed a tool to monitor end-user performance and how end users were doing.'' With new technology in Mercury Interactive's Topaz line, called Prism, ''I can monitor true end-user performance and have it reported to me [in] real time. I can see effectively whether they're having problems in their infrastructure or in the network, or if I'm having a problem,'' said Ziock.
Specifically, PlaceWare had a large customer that was reporting performance issues. ''Our systems performance monitors would always show us everything was all right, so we'd say 'It's got to be you,''' said Ziock. ''But if you're the customer, you don't want to hear that without proof.''
PlaceWare implemented Prism and started to monitor all customer usage on the service dynamically. ''We could see how quickly we were responding, and we were showing them this data, which found these network points were having problems from their side. Now they had data to bring back to their IT staff. This technology lets you know your customer -- when they're using the product, how it's being used, and what response time they're getting as they're using it,'' explained Ziock. ''There are not many technologies that allow you to know your customer. It makes my job of dealing with the customer easier. I don't have to do the 'yabba dabba do' of why it didn't work; we can tell them why. That's such an enormous benefit.''
All about quality
In addition to application monitoring, some organizations are utilizing tools that add application ''quality'' as a metric. Products in this area include Owl from IntegraSolv Corp., Tinton Falls, N.J.; the AppSight Suite from Identify Software, with its U.S. headquarters in Raleigh, N.C.; and IntegriTea from TeaLeaf Technology, San Francisco -- and each one takes a very different approach.
IntegraSolv's Owl monitors processes, components and connections, as well as the business flows supported by the application. Owl's modules include a Web-based GUI, Active Model, History, Data Collector and Agents. ''We collect telemetry data, not the payload type of data,'' said Arthur Frankstein, the firm's vice president, business development. ''We discover everything; when you see our graph on the screen, it's the actual manifestation of a real application network. We collect data about speed, count and transaction performance, and send it to a central server, then apply rules -- 'So if performance is such, then change the color of the map' -- to it. In the UI, there is a rich authorization and authentication model, so different people can have different views of that graph; business users can see a very high-level network, and operators can see something different.
''We're not going after infrastructure management, we're more business activity monitoring; we can see business granules, like orders,'' added Frankstein. ''I can't tell you why the database is slow, but I can tell you things are slowing down. Then you can go into your BMC or other database tool to diagnose the problem.''
The technology behind Owl is XML-based, and the product uses a ''SOAP-like'' protocol to talk to the applications, he explained. Frankstein said IntegraSolv is consulting with clients about Web services, and Owl already has a C# API.
Identify Software takes a different approach to application quality with AppSight, which is based on ''black-box'' technology. ''It's the same concept as a black box on a plane,'' explained Ran Gishri, vice president, product management. ''If the plane crashes, you can pull out the log and read it; we do the same thing for software. It's a software module you put on a client or server, and it captures the information to pinpoint a root cause of failure.'' Gishri said AppSight captures the live execution of applications on three levels: user (experience, transactions); system (database codes, network events, COM or COM+ events, processes), and code (C++, Visual Basic, Java and, very soon, any of the .NET-managed languages). ''All three levels are synchronized, so you can understand what happened as a result of a user action.''
Most AppSight customers use the product in a testing environment, where performance impact is not an issue, said Gishri, or in the production environment with a ''thin profile,'' which he said only has a 1% to 2% performance impact. If there is a problem in the production environment, a fuller version of AppSight can be installed. ''It's like taking a blood sample; we do the same with software,'' he said. ''Say you know the server is hanging every now and then. From our console you can start the black box and capture the problem, then close the black box.''
Like many vendors in this space, Identify Software is also positioning for .NET. ''We support almost the whole .NET framework, ASP.net and broader support for recording different Web services,'' said Gishri. ''We also support integration with Visual Studio .NET, so you can now write and open to Visual Studio .NET.'' (This capability is in beta.)
Another method for measuring the quality of the user experience comes from TeaLeaf's IntegriTea product, which monitors and plays back a user's Web session. For Dallas-based Mary Kay Inc., which processes 85% of the orders from its sales force over the Web, the ancillary benefits of this type of monitoring were unforeseen, extending now into Mary Kay's development process. ''Our site doesn't have a lot of performance issues, but we always look to get better,'' said Barry Bloom, chief architect of e-business. However, when he first encountered TeaLeaf, Bloom said he thought it was ''snake oil -- It seemed too good to be true, but it does everything they say it does.''
IntegriTea adds a new level of monitoring, said Bloom. ''There's hardware-level monitoring, OS monitoring, application-level monitoring; but there's not a whole lot of tools that monitor at the right level of detail. You want to know what the user experience is like. Tealeaf [IntegriTea] gives you a holistic view. Everything else is very atomic, saying how long a transaction took, or you're unable to connect to a database, or you got a timeout on this page, whereas TeaLeaf [IntegriTea] shows you what the user saw. To go from a timeout to that [type of information] is huge.''
Bloom provided an example of how IntegriTea improved the user experience for Mary Kay's Independent Beauty Consultants, of which there are 850,000 spanning 35 countries. Mary Kay's sales reps take orders from their individual customers and can process them through Mary Kay's Web site. Bloom explains that the site allows a sales force member to create and save orders. The button on that particular page used to say ''Click here to continue.'' Through IntegriTea, the company determined that the sales reps were not using that button because they wanted to save, not continue. With a simple change of the button to ''Click here to save that order,'' users ''can do one big order, so our average basket size now is 50 items. There was no bug or problem with the site, it was a usability thing we never would've found without TeaLeaf [IntegriTea],'' said Bloom. ''It's not groundbreaking, but it's an iterative way to make your site better by learning how the users are using it.''
IntegriTea captures every user session as it happens, detecting application failures in real time; it can then re-create and correlate the problem to the user. ''From the end-user perspective, IntegriTea is no different than a surveillance camera in a store; making a copy is really just a different kind of database, with very minimal overhead,'' said Bloom. Now IntegriTea is part of Mary Kay's development process, he said. ''During a release, we use IntegriTea in the testing process. If there's a problem, the developer can see exactly what happened; basically, you get a narrative from what occurred between the browser and the Web server.''
Mary Kay is also on the cutting edge of Web services. The e-business team there recently redesigned its order-entry system to fully leverage the .NET platform and the Windows 2000 DataCenter Server offering in the database tier and an ASP.NET Web interface. Other tools in Mary Kay's infrastructure management portfolio include ''NetIQ, Netcool, Compaq Insight Manager, Dell Open Manage and some custom tools. Tealeaf [IntegriTea] doesn't integrate natively, but it provides ways that are good enough,'' said Bloom.
Whether Web services are on the table in an organization today, or still on the whiteboard, the need for application monitoring and an understanding of the user experience are trends that will continue. ''With Web services, you're no longer beholden to only your own people; [you need] to provide computing capability to third parties. Monitoring is not a 'nice-to-have,' but a vital part of the solution set,'' said Xtremesoft's Dexter-Smith. ''If I'm providing data to you and it takes three minutes, but it only takes 30 seconds for my competitor [to provide data], clearly monitoring needs to become part of your business model. I've got to understand a lot more about what I'm providing, and how I'm providing it,'' he added.
So when the boss asks, ''How am I doing?,'' application monitoring tools will help provide the answers.