Emerging Technologies Spurring Need for Embedded Databases
- By George Lawton
Application service providers and mobile technologies said to be a perfect
fit for embedded systems; the right system can slash administration costs and
keep complexity away from end users.
Embedded databases are one of those behind-the-scenes technologies designed
to simplify the life of the programmer while remaining invisible to the end
user. Experts say such technologies, which have long served some small though
significant markets, are poised for substantial growth as the application service
provider (ASP) and mobile computing sectors grow quickly and add features to
create IT applications.
The traditional major market for embedded databases has been, and continues
to be, as an enabler of various enterprise applications. Analysts predict a
rapid expansion over the next five years as the technology matures and is specialized
to become a key technology for enabling a new class of mobile applications and
IT organizations like the embedded technology model, experts say, because the
database is hidden and confined to a single application. Thus, the systems are
generally much easier to manage than multi-purpose DBMS systems. A traditional,
high-end relational database can require the hiring of a DB administrator, with
an annual salary of $100,000-plus, to keep apps up and running. Analysts note
that while small- and mid-range-sized companies typically lack the funds to
hire such administrators, even the largest IT units are looking seriously at
embedded database technology as much for its cost-cutting capabilities as for
its technical benefits.
According to Norma Schroder, an analyst at GartnerGroup, a Stamford, Conn.-based
research firm, revenue for the embedded database market reached approximately
$450 million in 1999, while the entire database market garnered about $8 billion
in revenue. Ninety percent of the embedded database market is still in niche
vertical apps like accounting software, contact management or import/export
letter of credit generators. The mobile market has revenue of only about $60
million, and the appliance market is even smaller.
In most cases, corporate users do not know or care which database is embedded
into an application. "You buy the application, not the database," said Schroder.
"It is sort of like my not knowing the brand of carburetor that came in my BMW."
"Our customers say, 'I want you to be the plumbing in my house. I use it all
the time, but I don't want to think about it,'" said Paul Grabscheid, vice president
of strategic planning at InterSystems Corp., a Cambridge, Mass., maker of embedded
database systems. "In the embedded market the customer does not want the database,
they want the application."
InterSystems developed Caché, a high-end embedded database that the firm says
has been installed on more than 80,000 servers supporting about 3 million users.
Apps built with the product include thin clients, app servers in the middle
tier and database servers. Developers may choose to deploy these components
in a variety of configurations, depending on hardware capabilities and application
requirements, Grabscheid said.
"You find that the embedded database market tends to make all the costs visible.
If you go out and buy a contact management system, it bears all of the costs
of the developer and embedded database," Grabscheid said. "On the other hand,
if you are building a database application for a Fortune 500 company, it is
three to four years before you realize what the operational costs are, and by
that time you have done so many things that you lose sight of them all."
The glue behind the apps
Embedded databases today are primarily deployed inside various types of enterprises.
However, industry insiders say the widespread adoption of cellular telephone
data networks and the deployment of Internet appliances should expand the scope
and scale of embedded databases significantly over the next couple of years.
One of the most significant drivers of embedded databases may be the advent
of ASPs, an emerging offshoot of the outsourcing industry that promises to enable
corporate users to access applications from a wide variety of off-site, Internet-connected
InterSystems' Grabscheid and other suppliers of embedded database technologies,
such as Informix Software Inc., Sybase Inc. and PointBase, anticipate that rapid
growth of the ASP model can provide a major avenue for growth in their business.
"I see this driving the next order of magnitude or more of growth," Grabscheid
Observers noted that the ASP model utilizing embedded databases has started
to spread through the medical industry as hospitals and HMOs make information
available to patients via the Web. Such an application has the potential to
drive app usage from hundreds of doctors to hundreds of thousands of patients,
For example, IT developers at CareGroup HealthCare System, Boston, described
as the second largest integrated delivery system in the northeastern United
States, has developed CareWeb, a Web-based medical records-retrieval system
based on the Caché embedded database. The CareWeb system integrates the clinical
databases of two Boston-area medical centers, Beth Israel Hospital and Deaconness
Hospital, and is said to provide seamless, secure and almost instantaneous access
to records. CareGroup has annual revenue of $1.2 billion and a network of six
hospitals, 2,500 providers and 800,000 patients treated annually.
"They [doctors] often have a great need to get at these records, whether they
are in the office, at home or on the road," said John Halamka, CIO at CareGroup.
"Caché is the engine with which we store our clinical data and expose it in
a secure way over the Web."
The system maintains 90Gbs of clinical information in an online database. There
are 62 integrated applications that run on top of the clinical record system
for doing everything from reporting radiological results, to tracking stress
tests, drug prescriptions and clinical outcomes, Halamka said.
Halamka said the firm opted not to utilize traditional relational database
technologies because its medical data is not relational. "All of the medical
data is hierarchical and there are different events that happen to you. It is
very hard to reduce medical data to rows and columns," he noted.
The CareGroup system currently serves 900 simultaneous users across two Pentium
400-based servers. "Try doing that across Oracle or SQL," quipped Halamka. "Since
hierarchical data is so fast to retrieve, you only need cheap hardware that
costs $50,000, not [systems that can cost] multiple millions [of dollars].
"Ultimately," added Halamka, "patients will be given the ability to review
their records on the Web and to interact with their doctors via a secure messaging
Caché's internal structure is a multidimensional array that uses space
frugally. For example, with some types of data, such as patient records, Caché
is able to reduce the amount of space required to store data by 50% from that
required for a conventional relational database like Oracle. The real significance
is that the database can operate much faster if it can keep more of the data
in memory. Take the classic example of an online financial service's customer
record. One record can differ considerably from another, depending on the types
of investments a person makes and the services they use. Although traditional
bank accounts are fairly straightforward, financial apps are becoming more complicated
with the advent of online financial service firms that provide a variety of
services such as stock portfolios, bank accounts and bill-paying services.
"The Web is trying to drive people toward richer sorts of data," explained
InterSystems' Grabscheid. "What we are seeing, in general, is that people are
less interested in discrete applications and more [interested in] trying to
build much broader, comprehensive and sophisticated applications."
The mobile advantage
Mobile embedded databases enable developers to create applications for users
who might be only sporadically connected to a network, or who might have an
unreliable connection. "One of the main advantages of an embedded database is
that you have fast, up-to-date information on things you are interested in at
all times, independent of the signal quality and whether you are connected to
the 'Net or not," said Frank Rabe, general manager of the German operation of
embedded database software developer Empress Software, Greenbelt, Md.
But developers need some common hooks to simplify the task of integrating these
applications into the enterprise. Art Monk, vice president of marketing at Mountain
View, Calif.-based PointBase, said SQL support is important "because enterprises
are putting data out on mobile devices and they would like to be able to work
with one set of data access commands across multiple platforms. Rather than
have one on a Java application, another on a WinCE device and another on a client,
they would like to use straight SQL — which is what they are used to using in
the corporate environment."
PointBase was founded by Bruce Scott, co-founder of Oracle Corp. and Gupta Technology,
and co-architect of the first three versions of the Oracle relational database
management system. Scott said he founded the firm after he was convinced that
mobile and cellular technologies would soon be widely used as extensions to
corporate desktop systems.
The embedded database fray now includes Microsoft Corp., which last month (June)
started shipping SQL Server 2000 Windows CE edition with a footprint of 1Mb.
Observers said the Redmond, Wash.-based giant can become a player in the market
if the technology continues to improve at its current pace.
"One of the strengths of SQL Server is that it is well accepted because a lot
of developers are familiar with how to do ActiveX data objects," said Jeff Ressler,
lead SQL product manager at Microsoft. The software includes native OLE DB support,
which will help traditional SQL Server developers target the mobile device market.
The database supports a number of different data types, including Unicode and
images, and 128-bit encryption that can provide built-in security.
According to GartnerGroup's Schroder, the current mobile database market leader
is Emeryville, Calif.-based Sybase with its Sybase Anywhere product. Analysts
say the other major embedded database suppliers are PointBase, Oracle with its
OracleMobile system, Microsoft, Menlo Park, Calif.-based Informix Corp. and
its recently acquired Cloudscape DBMS, and IBM with its DB2 Universal Database
Satellite Edition. These and some smaller and less well-known suppliers are
concentrating most of their efforts in specific vertical markets, analysts say.
The dawn of the Internet appliance
The concept of Internet appliances runs the gamut from small devices to specialized
servers for mail and other applications. For example, the Empress database from
Empress Software Inc. is said to place no limit on storage capacity and can
store and retrieve virtually any type of data, from microsecond timestamps to
variable length text, to image and sound. Empress is used in embedded appliances,
such as office equipment, to track printing script language, protocols and log
entries. With this capability, a service company can check the status of a printer,
upload new configurations and troubleshoot problems. Within an organization,
a department manager can produce reports showing who printed specific materials.
At the high end of Internet appliance networking is Itron Inc., Spokane, Wash.
Itron has developed a complete data management architecture on top of Polyhedra
from Bellevue, Wash.-based Polyhedra Inc. Polyhedra is an active, real-time
fault-tolerant database designed for high-performance embedded database applications.
Itron is using the product as the database in its Fixed Network AMR system utility
"The client side has had a lot of focus on the size of the footprint for embedding
into things like Coke machines," said David Morse, managing director of Polyhedra.
"As those proliferate, you will see embedded databases on the server side. You
will not find us on the phone. We will be inside the base station that manages
the call for the phone."
The Polyhedra database is a main memory database that is limited only by the
size of available RAM. It has a facility that allows a backup database — stored
on a separate machine — to stay in lock step with the active database. If the
active database should crash for any reason, that backup would then become active.
The Polyhedra database provides Itron with data consolidation, network control
and fault tolerance. The system also uses the Polyhedra Database Interface to
send data directly to Oracle for billing and historical data processing.
Utilities can use Itron's Information Application Broker to create applications
that help automate the flow of data collection from gas, electric and water
meters. The data can be collected automatically via a wireless radio network
or on a weekly basis by roaming meter readers. A wireless hand-held device enables
a reader to collect data from 700 homes a day, while the car unit can scan more
than 5,000 homes per day. A fully automated network can scan the power usage
of all the homes on the network at multiple times throughout the day.
As utilities move toward deregulation, more frequent utility readings could
provide a firm with a competitive advantage. For example, if a particular power
company in the deregulated market cannot provide enough power to meet its customers'
demands, it is obligated to pay the local utility a hefty premium to cover the
"The utilities in some states will have to make up the shortfall if suppliers
are not supplying enough power in the grid," noted Bruce Angelis, software development
manager, mobile and network products at Itron. "The sooner they can do that,
the sooner they can get more money."
brew or not to brew?
While Java has not
yet gained wide acceptance in the embedded database market, a number of
industry insiders contend it will soon play a significant role. Norma
Schroder, an analyst at GartnerGroup, a Stamford, Conn.-based research
firm, said Java has not gained the same level of popularity in the embedded
market as it has in the enterprise application market for a variety of
reasons. "It is one of many [options] in a very fragmented market," she
"As the embedded devices
are getting more and more processing power and memory, I see more sophisticated
applications coming out on these devices," said John Kornatowski, president
and CEO at Empress Software Inc., Greenbelt, Md. "We see a fair amount
of that kind of development being done in Java. Right now, Java is mostly
on the client; but in another two years, there will be a lot more deployment
Java may play a much
larger role in the mobile device market, where it is already widely used
inside the browsers of most laptop computers. "At some point, Java is
going to become an important platform for mobile wireless devices and
laptops," said Art Monk, vice president of marketing at Mountain View,
Calif.-based PointBase. "It would make sense to write the database completely
in Java, so that when you link it up to another device, the whole thing
installs and runs easily. This means you can download the database to
anything that has a Java Virtual Machine [JVM]."
of Java is that it can help streamline the development of new applications
by allowing multiple parts of the development process to occur simultaneously.
"In the past, the device manufacturers would create the device, the OS
and then have the developers do the application. This was a very serial
operation," noted Monk. "With Java, they are decoupling that. They are
saying, 'All we need to do is guarantee that the hardware can support
a JVM.' This essentially cuts the development time in half."
The growing interest
in using the Linux operating system in embedded systems has prompted the
formation of the Embedded Linux Consortium (ELC), described as a vendor-neutral
trade association dedicated to advancing Linux-based solutions in embedded
applications. The consortium was unveiled this spring during the Embedded
Systems Conference held in Chicago.
Among the companies
joining the ELC is Centura Software Corp., Redwood Shores, Calif. Centura's
db.linux is an open source software (OSS) embeddable data management product
that enables developers to build applications for e-business and information
appliances. "The only thing we ask is that you call us for support," said
Scott Broomfield, CEO at Centura.
According to the company,
db.linux has a footprint of only about 200K of RAM, or about 40K if the
data access is read-only. The product is said to allow for records of
up to 32K in size; the number of fields within a record is limited only
by the maximum record size. Up to 256 files (record types) are allowed
While not a part of
the recently announced ELC, Greenbelt, Md.-based Empress Software Inc.
has also gotten into the Linux act. The company offers its Empress Embedded
Database Toolkit, Empress E-Commerce Database Toolkit and Empress Database
Toolkit on Linux. And in December 1999, Empress announced the availability
of its Empress Relational Database Management System on Cobalt Qube 2,
a network server appliance pre-configured with the Linux OS.