Business Intelligence Through a Clouded Crystal
- By Stephen Swoyer
CASTLE OF THE DARK CRYSTAL
- Crystal faces competition from Microsoft's successful SQL Server 2000 Reporting
Services offering and the Eclipse Foundation's Business Intelligence Reporting
- Some versions of Crystal are missing key features such as a centralized
or report repository, report lifecycle management capabilities and an enduser-
oriented Report Designer.
A reporting tool is a reporting tool is a reporting tool. From the perspective
of many enterprise programmers, it’s a truism. That’s because the
market for developer-oriented reporting tools has been dominated—perhaps
for as long as a decade—by Crystal Reports.
Things have changed a lot, however, and the Crystal of today isn’t necessarily
the 800-pound gorilla of developer-oriented reporting tools. It faces growing
competition on at least two fronts, principally from Microsoft’s successful
SQL Server 2000 Reporting Services offering, and from a nascent open-source
initiative, the Eclipse Foundation’s Business Intelligence Reporting Tool.
The former has already started to elbow its way into Crystal's bread-and-butter
Visual Studio .NET reporting business, and the latter—which just became
available in version 1.0 in June—threatens to do the same for J2EE, via
the hugely popular Eclipse IDE.
With a revamped Reporting Services on tap in SQL Server 2005, and with BIRT
development picking up steam, what’s in the cards for Crystal?
Crystal is promiscuous and ubiquitous
Like many codejockeys, Peter Tyler, a database programming consultant with The
Lutheran World Federation, started using Crystal Reports because it came bundled
with one of his shop's core business applications—ACCPAC, a shrink-wrapped
accounting application published by Best Software. Like many organizations,
Tyler’s organization expanded its use of Crystal over time so that it’s
now the outfit’s only software-based reporting tool.
“If we operate a database—apart from data stored in Excel, and
there is a lot of data quietly stored in Excel—we use Crystal Reports
8.5 to provide reports,” he says. “We feel that it is an extremely
rich reporting tool. It can report off a multitude of different data sources.
It can be very fast, too, if one knows how to tweak all the options. I can write
reports in [Crystal] off our accounting system…which return data much
faster than the application itself.”
Crystal Reports is one of the most prolifically bundled products in software
history. It has shipped with Visual Basic since version 3.0—released in
1993—and is bundled with development tools from Borland Software, BEASystems,
IBM and others. It’s been bundled with many BI offerings (such as Hyperion
Solution’s Essbase), and is extensively used as a default tool for operational
reporting (going directly against relational repositories) by a large number
It’s bundled with a bevy of other, non-BI applications, too, such as
ACCPAC. In fact, Crystal is so ubiquitous as to be downright promiscuous: Just
about every organization uses it somewhere, in some capacity. “Somewhere,
in someone’s toolset, you’re likely to come across Crystal Reports,”
says veteran industry-watcher Mike Schiff, VP of data warehousing and business
intelligence with consultancy Current Analysis.
One upshot of this is almost a de facto lock-in. Many organizations have enormous
investments in Crystal, Schiff says, in terms of reports that can’t easily
be migrated over to other platforms, as well as in Crystal programming expertise.
Developers have been working with Crystal (in some cases) for a decade or more,
and many business analysts are also familiar with its use.
Why would an organization heavily invested in Crystal want to switch to another
tool anyway? The chief reason is cost: Crystal Reports gives developers an easy
way to incorporate simple reporting capabilities into their apps, but in its
free, embedded form, isn’t designed to support large numbers of users,
nor to be used by non-developers, such as the business manager who wants to
create (or customize) her own report.
There’s a reason for this. Crystal’s strategy has been to seed
the market with Crystal Reports (through prolific bundling arrangements), encourage
adoption of and investment in Crystal and create opportunities to up-sell additional
Crystal products and services, says Business Objects’ James Thomas, product
Thomas didn’t—and probably wouldn’t—say as much, but
one way in which first Crystal, and now Business Objects, has tried to do this
is by leaving developers wanting more. To that end, the version of Crystal that’s
embedded with Visual Studio and other development tools is missing key features,
such as a centralized server or report repository, report lifecycle management
capabilities and an end-user-oriented Report Designer.
“The report design components that live within the IDEs essentially are
a lot of the objects for formatting, for data
access, for parameters, for formula languages,” Thomas says. “But
some functionality, including more of the complex
templates and sophisticated design elements, aren’t included in the IDEs.”
Business Objects sells them separately, however. The company provides a standalone
Crystal Reports Designer
component, which is aimed at business analysts and managers. Effective this
year, the company also sells a Crystal Reports Server bundle, which provides
a centralized repository for report data, along with report lifecycle management
capabilities. Then there’s Business Objects Enterprise (nee Crystal Enterprise),
high-end offering, which delivers fault tolerance (via clustering), support
for scheduling and other goodies.
The issue is that these up-sell products are available at prices some IT organizations
can’t afford. According to one prominent analyst, who declined to be named,
Microsoft’s free Reporting Services offering has forced Business Objects
to become more competitive on the low end (witness the new Crystal Reports Server,
available only for individual servers with up to four CPUs), but that hasn’t
affected the company’s high-end pricing.
As a result, some programmers are forced to make do with plain-vanilla Crystal
Reports—and its limitations. Such
constraints include a limit on the number of concurrent users (some codejockeys
report that performance issues crop up with as few as 15 concurrent users) and
strict adherence to tested best practices, such as eschewing the use of subqueries,
using OLEDB providers whenever possible, optimizing queries and filtering results.
Until recently, this was simply the status quo. However, Reporting Services
has changed that. And for some organizations that aren’t completely locked
into Crystal, or who aren’t users of Business Objects BI stack, Microsoft’s
enterprise reporting offering has emerged as a compelling alternative.
Reporting services rising
In the 18 months since it was first released, Reporting Services has already
won over some longtime Crystal developers, including Juan Arellano, a programmer
with a Latin American ISV and services firm who’s versed in Crystal. “I
think the strength of Crystal relies on the fact that it’s been the only
reporting tool in the market for so many years,” Arellano says. “Thousands
of companies have adopted it, and in many cases they aren’t willing to
assume the cost of changing to another.”
These days, Arellano says, he prefers working with Reporting Services whenever
he can. At the same time, he allows,
that’s not always possible because of customer resistance. “When
customers let me choose the reporting tool, I go for
Reporting Services,” he says. “But that’s not always the case—many
customers are already comfortable with Crystal,
and they are reluctant to change.” That’s the crux of the problem
for Microsoft, which in a rare role reversal, is trying to elbow its way into
a mature market that’s dominated by a popular entrenched player. Microsoft
has played to its
strengths, however, and made some progress, thanks to the ubiquity of Visual
Studio and the popularity of its SQL
Server 2000 database.
From the get-go, Microsoft has been careful not to position Reporting Services
as a competitor to Crystal, Brio and others on the BI reporting front, but as
a no-brainer solution for developers working with Visual Studio .NET.
“We’re saying, ‘Let’s just leverage the best development
tool out there. Let’s just integrate with Visual Studio so that they can
easily build reports,” says Tom Rizzo, group product manager for SQLServer,
during the run-up to the Reporting Services launch. “They can use the
debugging that they’re used to in Visual Studio, the data tools that they’re
used to in Visual Studio, so if you’re a Visual Studio customer, shame
on you if you don’t use [Reporting Services].”
There’s some evidence this strategy has been moderately successful. For
starters, says Microsoft, Reporting Services has been downloaded more than 140,000
times. Even if only a fraction of the users who download it are using and deploying
it in production environments, this still translates into a significant rate
There’s anecdotal evidence, too, that many experienced Crystal hands
like developer Arellano are at least open to the possibility of using Reporting
Services. Microsoft has published several prominent case studies, and its Reporting
Services USENET group typically buzzes with accounts of come-to-Reporting-Services
Take Brandon Osborne, a senior permanent consultant with integrator Encore
Consulting. In the on-the-beach, off-the-beach world of consulting, says Osborne,
he’s had an opportunity to work “with a little bit of everything.”
And even though Reporting Services still has its share of shortcomings compared
to Crystal, Osborne thinks Visual Studio developers should think about getting
hip to it.
“It may be in its infancy now, but when SQL Server 2005 comes out [you’ll]
be considerably ahead of the curve of other developers that thought Reporting
Services wasn’t going anywhere,” he argues, noting Encore already
has assisted customers in several successful Reporting Services rollouts. “One
of our clients in the financial sector utilizes C# and Reporting Services to
render the multitude of reports required in their application,” Osborne
says. “Considering that the bundled version of Crystal Reports only provides
five user licenses and Reporting Services doesn’t cause any licensing
constraints to those with a SQL Server license, Reporting Services was a no-brainer.”
Ironically, Osborne says, one of Reporting Services purported strong points—integration
with Visual Studio
.NET—is actually one of its weakest features, at least in comparison to
Crystal. “I like [Crystal’s] level of integration with Visual Studio
.NET,” he says. “If Reporting Services had this level of integration,
I’d be much more likely to recommend it,” he says while, noting
that Reporting Services doesn’t offer an easy way to embed reports in
.NET applications. “There should be a way to embed reports in a .NETsolution
without exporting the report to HTML format and displaying it in a Web browser
control,” Osborne says.
This will change in SQL Server 2005, Microsoft’s Rizzo says, while suggesting
a native Visual Studio control will make it easier for developers to embed reporting
capabilities—independent of Reporting Services’ SQL Server-based
underpinnings—in their applications. One upshot of this, Rizzo suggests,
is a thriving OEM market similar to Crystal’s own seeding strategy.
There’s reason to believe Rizzo isn’t telling tales out of school.
According to Encore’s Osborne, SQL Server 2005 Reporting Services looks
like a more serious competitor to Crystal. It boasts support for multidimensional
reporting, and also ships with a new WYSIWYG report design tool called Report
Builder. On top of
that, the revamped SQLServer 2005 Reporting Services retains the report lifecycle
management features and the zero
price tag of its SQL Server predecessor.
“Reporting Services for [SQL Server] 2000 leaves a lot to be desired,
but SQL Server 2005 hits the nail right on the head,” Osborne concludes.
“It’s considerably more advanced than the preceding version and
one of the key benefits is that it has been developed to be much more user friendly
than Crystal Reports.” The enterprise application development landscape
is far removed from .NET everywhere. Java and J2EE have an enormous and growing
presence. Yet, the J2EE IDE favored by the majority of developers has long been
ill-served on the reporting front. It’s still early, but that’s
BIRTh of a Java reporting tool
For a project as ambitious as Eclipse, the absence of an integrated reporting
facility seems like a serious oversight. After all, from the time IBM jumpstarted
Eclipse in late 2001, almost 3 years elapsed before reporting specialist Actuate
and the Eclipse Foundation announced the creation of a new top-level Business
Intelligence Reporting Tool (BIRT) project.
Meanwhile, Java reporting tools abound. There’s Jinfonet Software’s
JReports, JasperSoft’s Jasper, JFree and StyleReports, to name just a
few. Given the availability of these, and other free or pay-for-use open-source
tools, Eclipse’s lack of a canned reporting environment might not seem
like such a serious liability. To some extent, it hasn’t been.
According to an online poll conducted by the Eclipse Foundation, for example,
most J2EE developers prefer to build their own reporting facilities. Even BIRT
boosters acknowledge as much. “Basically, 85 percent of them are
doing roll-your-own [solutions],” says Mike Thoma, VP of marketing with
Actuate and Eclipse BIRTproject leader.
“They’re using Java and J2SE, and basically they have two recourses.
Their default now is to code [a custom reporting
tool] in Java. The second recourse is to buy commercial products. There’s
two kinds of those, the 100 percent pure Java ones and those that aren’t.”
From the perspective of many enterprise J2EE developers and the Eclipse Foundation,
the lack of a native Eclipse
reporting facility is a critical shortcoming. First, pay-for-use Java IDEs like
IBM’s Rational Application Developer for WebSphere Software, Borland’s
JBuilder, and BEA’s WebLogic Workshop all ship with Business Objects’
Second, it seems illogical: Eclipse is positioned as the most extensible of
frameworks, such that a plug-in or extension
for almost any conceivable language, feature, or capability can easily be incorporated
into it. That a plug-in for
such an essential service—reporting—has long been MIA strikes some
Eclipse codejockeys as inconceivable.
“I agree 100 percent…that embedded reporting is a key J2EE application
development pain-point,” wrote one
Eclipse user, a programmer with a government agency, in a post to the Eclipse
BIRT newsgroup. Like many codejockeys, this developer says he stumbled upon
BIRT while searching vainly for a functional Java-based reporting solution.
His problem? None of the open-source tools worked with the complex forms that
are de rigueur in the government sector.
In just 72 hours, however, a BIRT community contributor (and Actuate technologist)
responded with BIRTdesign code custom tailored for this programmer’s problem.
“Given what you’ve done…[and BIRT’s] support for pixel
accuracy in reproducing government forms, I now understand that BIRTwill support
such forms even though it is not its primary focus,” he wrote in response.
Talk about build to order. That’s the promise of the Eclipse project
and of open-source software, in general, and
even though the Eclipse Foundation announced BIRT1.0 (or release status 1.0)
only in June, a number of programmers are excited about its long-term prospects.
“Our first tests of BIRT have returned good results, basically because
it seems to be a good solution to our current
report tool problems,” says Martin Miguel López, a Java programmer
with TopGroup, an Argentine consulting and
software development firm. TopGroup is currently using JasperSoft’s Jasper,
says López, but has been frustrated by
that product’s shortcomings, which include a steep learning curve for
junior report developers, as well as performance
“We have a strong open-source orientation because the economic issues
of our country make our enterprise less
competitive if we choose commercial solutions,” López says. “So,
this means our report tool choices are minimal.”
Giving BIRT a look-see was a no-brainer in this instance, he says. “We
use Eclipse as our default IDE in Java projects, and we feel comfortable with
its perspective-based point of view,” he explains, noting that BIRT, which
has a report designer component, can be effectively used by both hardcore Java
coders as well as junior report developers.
In a lot of ways, López suggests, BIRT is already superior to most of
the existing Java reporting tools he’s looked at.
For example, its Report Designer GUI tool generates an XMLfile format that’s
proven to be more stable from version to version—a problem with Jasper.
There are other differentiators, too. “[R]eports are more flexible and
it seems to be so much easier to create complex mixed reports with graphs and
texts than with other tools,” López says. “The only thing
we found to be a big drawback of BIRT is that one of the most [important] features
[for] our customers…is the possibility to export reports to Microsoft’s
Excel format, and BIRTdoesn’t seems to have this feature.”
López concedes BIRTisn’t a perfect solution. He says TopGroup
would seriously consider Crystal Reports, if that product was open source, “We
feel very optimistic about BIRT, as it has become very mature in a very short
period since it was proposed as an Eclipse project,” he says.
When you look at the big players in the market for developer-oriented reporting
solutions, it’s tempting to picture things in terms of a pincer movement, with
Microsoft on one flank and BIRT (or another Java reporting solution) on the
other, slowly squeezing Crystal in the middle. To do that would be unrealistic,
however. (See related story, “Different
and divergent expectations for Crystal.”)
After all, as Business Objects’ Thomas points out, Crystal natively bridges
both the .NET and J2EE worlds, something that neither Reporting Services nor
BIRT can claim to do. More to the point, he says, Crystal will still be bundled
with Visual Studio .NET 2005, along with the three most popular commercial J2EE
“If you talk about reporting, you need to talk about any data source
and any development language, so it has to be both Java and .NET. It has to
work with whatever application server you have, not just WebSphere or Microsoft,”
he argues. “We still think there’s a need for an independent, platform-agnostic,
Java and .NET reporting platform. This is where you hit the wall with a pure-Microsoft
or a pure-Java approach. You have to have both.”
Of course, there’s some doubt among users about just how platform-agnostic
the new, Business Objects-stewarded
Crystal actually is, but few users would dispute that Crystal offers best-in-class
Thomas’ point regarding J2EE and .NET support is a legitimate one, says
Current Analysis’ Schiff. Simply by virtue
of its huge installed base, he argues, Crystal isn’t going anywhere.
What’s more, there’s plenty of room for all three players. “Alot
of people are still trained in Crystal, a lot of companies
have huge investments in Crystal reports, and Crystal works with both [J2EE
and .NET],” Schiff concludes. “So a more likely scenario is that
all three will coexist.”
and divergent expectations for Crystal
More on ADTmag.com
BI tool keeps
Independence Air competitive
By Lana Gates
Eclipse brings open source
to BI market
By ADT Staff
The SQL Server 2005 paradigm
By Stephen Swoyer
PHOTO BY BRUCE AYRES