It's All in the Cache -- ADTmag

It's All in the Cache

There are advantages and disadvantages to today's application-tier caching techniques. Consider your data caching options for improving data-intensive application performance.

By Marie-Anne Neimat
March 28, 2007

Relational database management systems (RDBMSs) are the standard data repositories and transaction engines of the business world. They are deployed typically on dedicated servers and I/O subsystems configured for optimum performance of the database management system (DBMS). Applications that access these databases run on different servers and access the databases through their client/server interfaces or indirectly through application servers or other layers of abstraction.

Various caching techniques have been devised to cache data in the application tier, close to applications. Caching is used to avoid repeated network trips from the application tier to the database server and/or to avoid repeated mappings between an application's data model and the database's relational model. This reduction in network traffic diminishes greatly the latency of data access. Reducing frequent requests to the back-end DBMS has the added advantage of improving overall throughput of the DBMS. Reducing traffic between application tier nodes and the back-end database server can also reduce network congestion.

Before we take a look at various application-tier caching techniques on the market and the advantages and disadvantages of each approach, let's look at what an application is trying to accomplish by caching data in the application tier. The main motivation of course is improved application performance; however, there are other considerations as well, including a read-only versus an updatable cache, currency of the cache, distribution of the cache on multiple nodes, data model and query capability, persistence in the application tier, and availability. Let's look at the details of each.

Read-only versus updatable cache. The most fundamental question when assessing the suitability of a caching technology for a given application is whether the application needs to update cached data, or whether the cached data will be limited to read-only access by the application.

Currency of the cache. The content of a cache can get stale over time if cached data is updated in the back-end DBMS but not refreshed in the cache. There are various techniques that are used to maintain cache currency. Some rely on periodic invalidations of the cache, some rely on periodic refreshes, some rely on instantaneous refreshes, and others rely on instantaneous invalidations when the data in the back-end database is updated. For many applications content that is slightly stale is acceptable. However, for other applications, this condition isn't acceptable; hence, the ability of the caching technology to keep the cache current will affect its suitability for a given application.

Distribution of the cache on multiple nodes. Another consideration is whether the application itself is distributed on multiple application-tier nodes. If it is, then you can assume that there will be multiple caches, with one cache residing on each node. An important consideration then is whether the same data items may appear in many nodes, or whether each data item can be in one node only at any point in time. Multiple copies of the same data item complicate cache coherence, especially if the caches are updatable.

Data model and query capability. Considering the query capabilities required from the cache is also important. The application may require the full functionality of SQL to have access to its rich query capability. Or it may be managing Java or C++ objects with a need for some query capability or no query capability. In the latter case, access to objects may be through their object identifiers with no need for search capability.

Persistence in the application tier. Another consideration when evaluating caching technologies is whether there is a need for persistence in the application tier. In other words, is the cache strictly temporary, and can it therefore disappear if the cache node goes down? Related to the need for persistence are the size of the cache and the availability requirements of the cache.

Availability. An important factor when considering different caching technologies is availability. For example, should access to data continue to be available if the back-end DBMS goes down or if access to the network is impaired? What if the back-end DBMS has a high-availability capability? Should the cache be able to failover to a surviving node?

It is clear that many factors influence the appropriateness of a caching technology to a given application.

Caching Techniques
There are several caching techniques that have been developed and are available on the market. Let's take a look at some of them.

Query result caches are managed by special software that typically hides the presence of the cache from the application. Under this scenario, the caching software automatically saves the results of queries that are submitted to the DBMS. A cache hit is recognized and serviced from the cache if a query is an identical match to a previously submitted query, including identical values of parameters.

The advantage of such caching is that it is simple, and it caters to access scenarios where the same query is likely to be submitted repeatedly. To handle the currency of the cache, some of the DBMSs on the market today provide a notification mechanism to alert the application-tier software that certain values in the cache should be invalidated because they were updated on the back-end DBMS. These caches are limited in scope as they cannot handle query processing on the content of the cache. Further, these caches are not updatable, and they are not distributed.

Object-relational mapping tool (O/R mapping tools) caches, such as Oracle TopLink, hide relational databases from object-oriented programmers by providing transparent mapping between objects and relational data. Once relational data is mapped to an object representation, it may be cached by the O/R mapping tool until it is no longer needed or until it becomes stale. Caching by O/R mapping tools is a common technique to avoid the expensive mapping between the programming language's object model and the database's relational model.

These tools rely on various techniques to keep the caches current. The techniques include invalidation of the cache at the end of a transaction or relying on the back-end DBMS's invalidation service to invalidate stale data items. The caches can be updatable and the O/R mapping tools map updates to objects to updates to relational data. While multiple such caches may be set up on multiple nodes, each cache is aware of its own state only.

In object caches the concept of "caching" is somewhat of a misnomer because objects that end up in these caches are not necessarily subsets of objects that are stored elsewhere. These caches are repositories of objects that are independent of the object's origins. They aren't typically transparent to applications. Applications put, get, insert, and delete objects from the caches. There are a few products on the market that offer such caches, and they vary in the level of functionality they support. The caches may be strictly memory resident, or they may be backed up to disk or to a DBMS. Some products provide concurrency control, some provide transparent distribution over multiple nodes in a network, and some provide high availability. The query capability of these products is limited, as they don't support SQL functionality.

Object-oriented DBMSs are often implemented as client/server architectures with some of the DBMS logic—namely the client—residing in the application tier and some residing on a shared server. The client architecture is typically main memory based. In other words, the DBMSs come with built-in caching and built-in cache distribution.

For applications implemented in Java or C++ they can also save on the expensive mapping between relational data and object representations. However, object-oriented DBMSs don't conform to an agreed-upon standard, and they don't offer the richness and completeness of relational DBMSs. Further, most business data is managed by relational DBMSs, thus putting object-oriented DBMSs at a main disadvantage.

In-memory RDBMSs are architected to take full advantage of memory residence of the data to optimize data layout and access, thus reducing the number of instructions required to execute a DBMS request. In addition to being deployed as stand-alone databases, in-memory relational DBMSs may be used to cache subsets of a back-end relational DBMS.

The fact that they have identical models, semantics, and APIs makes such caching much more natural and has the potential of being seamless. Objects that are cached are tables or table fragments. The content of the cache may be preloaded before the application starts, or it may be brought into the cache on demand. Cached objects may be updated in the cache or in the back-end database. For example, Oracle TimesTen is an in-memory DBMS that may be used as a cache to the Oracle Database.

Data caching in the application tier is an effective technique that may be used by data-intensive applications to improve their overall performance. Several factors that an application developer should consider before deciding on a caching strategy and underlying technology have been presented here, along with several caching technologies that may be adopted to speed up data access within an application.