In-Depth

Emerging Technologies Spurring Need for Embedded Databases

• By George Lawton
• January 1, 2001

Application service providers and mobile technologies said to be a perfect fit for embedded systems; the right system can slash administration costs and keep complexity away from end users.

Embedded databases are one of those behind-the-scenes technologies designed to simplify the life of the programmer while remaining invisible to the end user. Experts say such technologies, which have long served some small though significant markets, are poised for substantial growth as the application service provider (ASP) and mobile computing sectors grow quickly and add features to create IT applications.

The traditional major market for embedded databases has been, and continues to be, as an enabler of various enterprise applications. Analysts predict a rapid expansion over the next five years as the technology matures and is specialized to become a key technology for enabling a new class of mobile applications and Internet appliances.

IT organizations like the embedded technology model, experts say, because the database is hidden and confined to a single application. Thus, the systems are generally much easier to manage than multi-purpose DBMS systems. A traditional, high-end relational database can require the hiring of a DB administrator, with an annual salary of $100,000-plus, to keep apps up and running. Analysts note that while small- and mid-range-sized companies typically lack the funds to hire such administrators, even the largest IT units are looking seriously at embedded database technology as much for its cost-cutting capabilities as for its technical benefits.

According to Norma Schroder, an analyst at GartnerGroup, a Stamford, Conn.-based research firm, revenue for the embedded database market reached approximately $450 million in 1999, while the entire database market garnered about $8 billion in revenue. Ninety percent of the embedded database market is still in niche vertical apps like accounting software, contact management or import/export letter of credit generators. The mobile market has revenue of only about $60 million, and the appliance market is even smaller.

In most cases, corporate users do not know or care which database is embedded into an application. "You buy the application, not the database," said Schroder. "It is sort of like my not knowing the brand of carburetor that came in my BMW."

"Our customers say, 'I want you to be the plumbing in my house. I use it all the time, but I don't want to think about it,'" said Paul Grabscheid, vice president of strategic planning at InterSystems Corp., a Cambridge, Mass., maker of embedded database systems. "In the embedded market the customer does not want the database, they want the application."

InterSystems developed Caché, a high-end embedded database that the firm says has been installed on more than 80,000 servers supporting about 3 million users. Apps built with the product include thin clients, app servers in the middle tier and database servers. Developers may choose to deploy these components in a variety of configurations, depending on hardware capabilities and application requirements, Grabscheid said.

"You find that the embedded database market tends to make all the costs visible. If you go out and buy a contact management system, it bears all of the costs of the developer and embedded database," Grabscheid said. "On the other hand, if you are building a database application for a Fortune 500 company, it is three to four years before you realize what the operational costs are, and by that time you have done so many things that you lose sight of them all."

The glue behind the apps

Embedded databases today are primarily deployed inside various types of enterprises. However, industry insiders say the widespread adoption of cellular telephone data networks and the deployment of Internet appliances should expand the scope and scale of embedded databases significantly over the next couple of years. One of the most significant drivers of embedded databases may be the advent of ASPs, an emerging offshoot of the outsourcing industry that promises to enable corporate users to access applications from a wide variety of off-site, Internet-connected devices.

InterSystems' Grabscheid and other suppliers of embedded database technologies, such as Informix Software Inc., Sybase Inc. and PointBase, anticipate that rapid growth of the ASP model can provide a major avenue for growth in their business. "I see this driving the next order of magnitude or more of growth," Grabscheid said.

Observers noted that the ASP model utilizing embedded databases has started to spread through the medical industry as hospitals and HMOs make information available to patients via the Web. Such an application has the potential to drive app usage from hundreds of doctors to hundreds of thousands of patients, insiders say.

For example, IT developers at CareGroup HealthCare System, Boston, described as the second largest integrated delivery system in the northeastern United States, has developed CareWeb, a Web-based medical records-retrieval system based on the Caché embedded database. The CareWeb system integrates the clinical databases of two Boston-area medical centers, Beth Israel Hospital and Deaconness Hospital, and is said to provide seamless, secure and almost instantaneous access to records. CareGroup has annual revenue of $1.2 billion and a network of six hospitals, 2,500 providers and 800,000 patients treated annually.

"They [doctors] often have a great need to get at these records, whether they are in the office, at home or on the road," said John Halamka, CIO at CareGroup. "Caché is the engine with which we store our clinical data and expose it in a secure way over the Web."

The system maintains 90Gbs of clinical information in an online database. There are 62 integrated applications that run on top of the clinical record system for doing everything from reporting radiological results, to tracking stress tests, drug prescriptions and clinical outcomes, Halamka said.

Halamka said the firm opted not to utilize traditional relational database technologies because its medical data is not relational. "All of the medical data is hierarchical and there are different events that happen to you. It is very hard to reduce medical data to rows and columns," he noted.

The CareGroup system currently serves 900 simultaneous users across two Pentium 400-based servers. "Try doing that across Oracle or SQL," quipped Halamka. "Since hierarchical data is so fast to retrieve, you only need cheap hardware that costs $50,000, not [systems that can cost] multiple millions [of dollars].

"Ultimately," added Halamka, "patients will be given the ability to review their records on the Web and to interact with their doctors via a secure messaging application."

Caché's internal structure is a multidimensional array that uses space frugally. For example, with some types of data, such as patient records, Caché is able to reduce the amount of space required to store data by 50% from that required for a conventional relational database like Oracle. The real significance is that the database can operate much faster if it can keep more of the data in memory. Take the classic example of an online financial service's customer record. One record can differ considerably from another, depending on the types of investments a person makes and the services they use. Although traditional bank accounts are fairly straightforward, financial apps are becoming more complicated with the advent of online financial service firms that provide a variety of services such as stock portfolios, bank accounts and bill-paying services.

"The Web is trying to drive people toward richer sorts of data," explained InterSystems' Grabscheid. "What we are seeing, in general, is that people are less interested in discrete applications and more [interested in] trying to build much broader, comprehensive and sophisticated applications."

The mobile advantage

Mobile embedded databases enable developers to create applications for users who might be only sporadically connected to a network, or who might have an unreliable connection. "One of the main advantages of an embedded database is that you have fast, up-to-date information on things you are interested in at all times, independent of the signal quality and whether you are connected to the 'Net or not," said Frank Rabe, general manager of the German operation of embedded database software developer Empress Software, Greenbelt, Md.

But developers need some common hooks to simplify the task of integrating these applications into the enterprise. Art Monk, vice president of marketing at Mountain View, Calif.-based PointBase, said SQL support is important "because enterprises are putting data out on mobile devices and they would like to be able to work with one set of data access commands across multiple platforms. Rather than have one on a Java application, another on a WinCE device and another on a client, they would like to use straight SQL — which is what they are used to using in the corporate environment."

PointBase was founded by Bruce Scott, co-founder of Oracle Corp. and Gupta Technology, and co-architect of the first three versions of the Oracle relational database management system. Scott said he founded the firm after he was convinced that mobile and cellular technologies would soon be widely used as extensions to corporate desktop systems.

The embedded database fray now includes Microsoft Corp., which last month (June) started shipping SQL Server 2000 Windows CE edition with a footprint of 1Mb. Observers said the Redmond, Wash.-based giant can become a player in the market if the technology continues to improve at its current pace.

"One of the strengths of SQL Server is that it is well accepted because a lot of developers are familiar with how to do ActiveX data objects," said Jeff Ressler, lead SQL product manager at Microsoft. The software includes native OLE DB support, which will help traditional SQL Server developers target the mobile device market. The database supports a number of different data types, including Unicode and images, and 128-bit encryption that can provide built-in security.

According to GartnerGroup's Schroder, the current mobile database market leader is Emeryville, Calif.-based Sybase with its Sybase Anywhere product. Analysts say the other major embedded database suppliers are PointBase, Oracle with its OracleMobile system, Microsoft, Menlo Park, Calif.-based Informix Corp. and its recently acquired Cloudscape DBMS, and IBM with its DB2 Universal Database Satellite Edition. These and some smaller and less well-known suppliers are concentrating most of their efforts in specific vertical markets, analysts say.

The dawn of the Internet appliance

The concept of Internet appliances runs the gamut from small devices to specialized servers for mail and other applications. For example, the Empress database from Empress Software Inc. is said to place no limit on storage capacity and can store and retrieve virtually any type of data, from microsecond timestamps to variable length text, to image and sound. Empress is used in embedded appliances, such as office equipment, to track printing script language, protocols and log entries. With this capability, a service company can check the status of a printer, upload new configurations and troubleshoot problems. Within an organization, a department manager can produce reports showing who printed specific materials.

At the high end of Internet appliance networking is Itron Inc., Spokane, Wash. Itron has developed a complete data management architecture on top of Polyhedra from Bellevue, Wash.-based Polyhedra Inc. Polyhedra is an active, real-time fault-tolerant database designed for high-performance embedded database applications. Itron is using the product as the database in its Fixed Network AMR system utility meter-reading application.

"The client side has had a lot of focus on the size of the footprint for embedding into things like Coke machines," said David Morse, managing director of Polyhedra. "As those proliferate, you will see embedded databases on the server side. You will not find us on the phone. We will be inside the base station that manages the call for the phone."

The Polyhedra database is a main memory database that is limited only by the size of available RAM. It has a facility that allows a backup database — stored on a separate machine — to stay in lock step with the active database. If the active database should crash for any reason, that backup would then become active. The Polyhedra database provides Itron with data consolidation, network control and fault tolerance. The system also uses the Polyhedra Database Interface to send data directly to Oracle for billing and historical data processing.

Utilities can use Itron's Information Application Broker to create applications that help automate the flow of data collection from gas, electric and water meters. The data can be collected automatically via a wireless radio network or on a weekly basis by roaming meter readers. A wireless hand-held device enables a reader to collect data from 700 homes a day, while the car unit can scan more than 5,000 homes per day. A fully automated network can scan the power usage of all the homes on the network at multiple times throughout the day.

As utilities move toward deregulation, more frequent utility readings could provide a firm with a competitive advantage. For example, if a particular power company in the deregulated market cannot provide enough power to meet its customers' demands, it is obligated to pay the local utility a hefty premium to cover the shortfall.

"The utilities in some states will have to make up the shortfall if suppliers are not supplying enough power in the grid," noted Bruce Angelis, software development manager, mobile and network products at Itron. "The sooner they can do that, the sooner they can get more money."