In-Depth

The transition of OLAP

• By Rich Seeley
• July 1, 2001

Even the term "OnLine Analytic Processing" may be outdated. Like "live via satellite" and "direct dial telephone," "online" is becoming an old-fashioned term. This could help explain why some companies offering OLAP use the term sparingly, if at all.

Since Microsoft entered the market in late 1998 and became a leading vendor in the space, it has renamed OLAP. Having enjoyed success with its OLAP Services component of SQL Server 7.0 and in introducing its new version, Microsoft boldly stated: "SQL Server 2000 extends and renames the former OLAP Services component, now called Analysis Services." It is probably too early to say if Analysis Services will become synonymous with OLAP, but as one analyst points out, Microsoft has a way of establishing de facto standards and naming conventions.

Whether it does for OLAP what it did for CP/M remains to be seen, but Microsoft is having an impact on the market. Two years after it became an OLAP vendor, Microsoft was ranked as the third largest OLAP vendor by the London-based Business Intelligence Ltd. research company last year. Microsoft surpassed Oracle Corp. and MicroStrategy Inc., which were ranked fourth and fifth.

Business Intelligence's The OLAP Report finds an OLAP world where there are still many players, despite startup failures and acquisitions, with no single company taking over leadership—yet.

According to the research company's report, "The OLAP market currently has no dominant players, and we do not expect any single vendor to dominate it within the next two years. Neither Hyperion Solutions nor Cognos, the two largest market share holders in 2000, can be regarded as dominant, and several of the top ten had a falling market share in 1999 and 2000. The largest potential player [Microsoft] did not enter the market until the end of 1998, and its effects are already being felt. But it will not have the leading revenue market share until at least 2002."

With 2002 still months away, this article looks at the current state of OLAP, where the technology, by whatever name, is headed, and how it is currently being implemented and used.

Whither OLAP?
The change in OLAP technology and the marketplace is part of a trend that analysts and consultants have been noting for the past two years. "What happened is that OLAP is being subsumed into other products," said Bill Inmon, consultant, author and generally acknowledged creator of the OLAP specification. "Some of the smaller vendors have either merged or gone away. You still have Brios, Cognos, MicroStrategy and a few others. But for the most part, the larger vendors have incorporated OLAP into their product lines."

Nigel Pendse, co-author of The OLAP Report, believes the hardest hit by this changing market are the OLAP server vendors, who have faced the toughest competition and the most dramatic changes.

"If you look at the results, you will see OLAP server vendors are suffering, whereas OLAP client tool vendors are doing pretty well," he said. "The small OLAP server vendors are in real trouble. But even the big ones, like Hyperion or MicroStrategy, are now showing declines in licensing fees."

Enter Microsoft
What brought about the change from a wide open, free-for-all market with plenty of room for small specialty and boutique vendors to a more competitive, unforgiving arena where only the strongest and toughest survive? Both analysts agree this change was not just normal growing pains.

"The OLAP vendors' role is changing, much to their chagrin," said Inmon. "They once had the marketplace to themselves, but it is now being taken over by Microsoft, SAP and, for that matter, PeopleSoft."

"[Microsoft] entered the market at the end of 1998," Pendse said. "They are already the market leader." Microsoft, which has bundled their OLAP server with SQL Server, has succeeded not by marketing muscle, said Pendse, or by leveraging the popularity of SQL Server. "I think they've done a poor job marketing it."

Pendse claimed that in many cases, having SQL Server in the bundle is irrelevant. "They buy it for the OLAP server, not because of the SQL Server," he said. "Plenty of Oracle and DB2 sites are buying Microsoft's OLAP server and they may or may not even install SQL Server."

Asked what is driving sales, if not marketing or SQL Server, Pendse said: "It performs well, it's scalable, it's functional, it's easy to use, it has a lot of third-party front-end support and it's good quality. There are very few bugs compared to its competitor."

Another factor may be the popularity of the Windows OS, "not just for the end-user interface, but for the server platform," said Pendse. "The latest quantitative data I have says that about two-thirds of OLAP servers are installed on either NT or 2000."

Inmon sees the OLAP market changing in other ways, as well. As it matures, and as the bigger vendors incorporate OLAP into their larger product lines, OLAP is becoming more sophisticated and more tied to analytical applications.

"Some interesting analytical apps are appearing," Inmon observed. "The OLAP vendors that are going to succeed are those that cater to analytical applications and go out of their way to make sure that the OLAP applications work well with them."

The most well known analytical application is CRM, said Inmon, but there are others that OLAP vendors need to be addressing, including Churn Management for customer retention analysis, Yield Management for dynamic pricing in an online mode and Elasticity Analysis for determining the optimal sales price for a retailer.

Implementation ain't easy
Whether OLAP is wanted for simple report processing, fancy pie charts or graphics, or sophisticated analytical applications, it is not easy to implement. One reason for this is that OLAP will not work the way it is intended to unless the underlying databases and corporate systems are fully integrated.

"People have discovered that if they do not have the foundation beneath OLAP, they do not get their money's worth," Inmon said.

Chuck Ballinger, information analyst for Avista Energy, Spokane, Wash., an energy company serving approximately 32,000 square miles in the American Northwest, had to deal with this issue while implementing a new HRIS application.

Before the OLAP reporting from the main data warehouse, which receives data from throughout the company, would work optimally, Ballinger needed to integrate the data warehouse to local databases.

Avista Energy runs a three-tiered environment, with the server farm and most of the applications running in Oracle. The customer information system and the billing and finance back-end systems tied to it are all DB2, which is housed in a data center in Sacramento, Calif. The local Oracle servers are housed in the computer room in a complex in Spokane and are tied remotely to the DB2 data warehouse. Translations and transformations of data from the various sources are done using an ETL system from Data Junction Corp., Austin, Texas.

"Because we store all our mainframe data in DB2 and our HRIS system writes out to Oracle databases, we had a problem getting data from one database to the other on a consistent basis," Ballinger remembered. "Now, we use Data Junction to do just that."

With the integration in place, after every payroll run, the system automatically exports the necessary data into DB2, Ballinger explained. The OLAP reporting then provides reports for finance, accounting, and federal and state regulatory commissions.

Prior to the implementation of the new system, it could sometimes take up to two weeks to produce critical financial reports for company officers and executives. "We can now produce those reports with the OLAP system two days after closing," Ballinger said. "Usually within two days of close, managers and department supervisors have access to the reports that will give them their labor distribution, their cost and their expenses for that month."

Getting from two weeks to two days turnaround may be seen now as a great benefit to the company's managers, but the new system was not an easy sell at first.

"The biggest hurdle was the cultural one," Ballinger recalled. "To get things turned around in a rapid order, to get everybody [to agree] that we needed to do this for the health of the company. We're sitting on valuable information and it doesn't do anybody any good if three weeks into the next month you discover, hey, we shouldn't have been doing this for the past three weeks."

Like most successful large-scale implementations, Ballinger credits buy-in from upper management with making the project possible. "Our VP of finance was very keen on providing this," Ballinger said. "He was the major driver and also the major benefactor of the system."

Managers have the client version of the data warehouse software installed on their Windows NT-based terminals, so they can view the reports the OLAP system creates. "They can look all the way down to the specific employee level with costs, allocations, distributions and how the employees' time is spent broken down," Ballinger said.

Unlike many applications that are designed to be used internally by the system or used by highly trained IT staff, OLAP services are used by end users. And that means the headaches do not stop at deployment.

When implementing a new OLAP system, Ballinger believes it would be easier to restrict access to power users, at least in the beginning. "There are a lot of concepts that they need to be aware of," warned Ballinger. "And when you start talking about developing a cube and doing things in three dimensions, you tend to leave the average user in the dust unless you can lead them by the hand and do a lot of training to get them to understand exactly what they are modeling and why they are modeling it that way."

Pendse believes that this has been a problem with OLAP systems for a long time, one that vendors are now trying to address. "I think the trend is to make the functionality easier to use rather than just to be more functional," he said. "Older products like Oracle Express were very functional. They were just too hard to use."

Another common headache Ballinger had to face was the discrepancy between user expectations and reality. Even though the system may be crunching four million lines of data to produce a report, Ballinger said many users are impatient for the answer. "It is not uncommon for it to run five and six hours at a time," Ballinger said. "The users had the perception that, like a demo, you click the button and poof, up pops the answer. And they become dismayed when they click on the button and two hours later they still haven't gotten the answer."

Ballinger is not alone, according to Inmon, and it is not only the end users who sometimes have unrealistic expectations.

"Even if they get through the issues of volumes of data, then they hit the issues of integration of data," Inmon said. "It's one thing to say I can process a large volume of data; it's another thing to say that the data that I have is integrated and is of a high quality. So, it's like peeling an onion back. You hit one layer, then you hit another layer, and you discover that the glitzy presentation that the OLAP vendor made really did not tell you about the problems at all."

Some of the problems companies run into, Ballinger said, are really a result of too little power in the hardware. "A lot of attempts fail because they attempt to either put it on an existing box that's already doing a lot of work in their organization, or they underestimate just exactly what kind of horsepower is needed," Ballinger said. "It's really hard to get any evaluation of your data from a vendor's PC during a demo because you're not running the real data."

Ballinger's company is running high-end IBM SP 6000s. Their servers run at 800 MHz or better and each has at least half a Gig of memory. In some cases, the company connects three or four CPUs so the tasks required for large cubes can be shared among them. "The hardware has finally come along now to actually help the software end of it, to give it a real boost," Ballinger said. "You no longer could gain much with software alone because of the sheer number of computations and the volume of data that you had to handle."

Analyze this: Maturing OLAP
Wherever OLAP is going, it is not likely to return to the days when boutique vendors dazzled end users in corporate marketing and sales departments by showing how yesterday's gross sales figures could be formatted into colorful bar graphs and pie charts. Whether or not Microsoft's Analysis Services name sticks, end users are beginning to demand greater functionality and more practical applications.

As Inmon sees it: "OLAP's gone to the next step, where users say, 'OK, now that you can do a pie chart, tell me something. Help me draw some conclusions.'"

For example, with the latest technology, the output from OLAP starts with analytical applications that are crunching numbers on customer turnover to produce reports that can be used for churn management.

"It's no longer sufficient just to make pretty little pictures," Inmon said. "Those pictures have to say something related to the business of the corporation."

And before investing in OLAP, Pendse recommended that IT departments make sure the software fits the end users' needs. A tool designed for the financial industry may require painstaking customization for a retail operation, and still may not offer the functionality and usability of an off-the-shelf product designed for tracking in-store sales.

"Don't just go out and buy technology for the sake of buying it," Pendse warned. "The products differ quite a lot. And it is silly to buy technology because you think it is good, without knowing if it is suitable for you. Too many IT people just buy what they think is good technology without knowing what they are going to do with it."

IT professionals' jobs may be made easier as Microsoft, Oracle, SAP and PeopleSoft add OLAP capabilities to the ERP, CRM, database, data warehouse and data mart systems that provide the foundation for what Inmon terms the Corporate Information Factory, which can support a wide range of analytical applications. OLAP, or whatever it ends up being called, may then evolve from the technology buzzword of 1995 to just another functional cog in the information system.