Enterprise JavaLarge-Scale Servlet Programming
THE JAVA SERVLET API and Java Server Pages (JSPs) are a great help in developing high-performance, server-side Java programs for the Web. However, there are considerations that have to be taken into account when dealing with very large, high-traffic Web sites that can affect the design of your servlets. We examine the key scalability issue of storing client data on the server and some approaches for making your servlets perform in a high-traffic environment.
What is Session Data, Anyway?
Generally a session in a Web environment is a set of interactions between a user/client's Web browser and a particular Web server. The session starts with the first URL call to that Web server and ends when either the Web server ends the session, "times out" the session, or when the user closes his/her browser. Session data is information the user provides that is used across multiple screens prior to permanently saving the information. The distinction between session data and transaction data is that session data is temporary—only for use across a set of linked pages—and transaction data is placed in permanent storage. Session data is often converted into transaction data at the end of a set of Web pages (when a user chooses to "commit" a transaction, or "check out" a shopping cart).
Consider the following scenario: Our user, Rob, types the URL for his favorite (fictional) wine shopping site, www. winesRus.com. Rob browses for wines to buy and adds some to his shopping cart. All of the interaction with the site takes place in a single session controlled by the www.winesRus.com Web server. The information about the items Rob purchases, his address, and any other information Rob provides is kept as session data as he navigates from screen to screen shopping for more wines. The site prices his purchases and asks him to confirm them. When Rob confirms, the session data is then used to execute a purchase transaction and the data becomes permanent.
Servlets correspond one-on-one with a particular HTTP URL, much as each URL in traditional Web programming would have its own CGI script. For example, http://winesRus/servlet/purchaseWinesServlet script corresponds with http://winesRus.com/servlet/purchaseWinesServlet.
Just like CGI scripts, servlets are by themselves stateless. Also like CGI scripts, servlets get client data from the HTTP parameters and HTML forms. On a particular application server, a single instance of each servlet class handles all doGet() and doPost() requests for its particular URL. Each HTTP request is handled on a unique thread running the service() method of that instance. Because each servlet instance is a shared resource, you can't store the client session data (such as a customer's shopping cart) in the servlet itself. Session data must be stored outside the servlet.
There are several approaches to storing session data that we examine in this article. How session data is stored is a key element effecting the scalability of a Web site. We look at the pros and cons of each and make recommendations as to the best ways to proceed in particular cases.
Application Server Facilities
Probably the easiest way to store session data is by using the built-in facilities that are provided by the application server for that purpose.* The Sun Java Servlet API introduced a way to store session information in version 2.0. This is accomplished through the HTTPSession interface, which provides methods for storing, finding, and removing objects from a dictionary based on keys. An application server provides a class implementing HTTPSession.
The most important methods of HTTPSession are:
public abstract void putValue(java.lang.String param1, java.lang.Object param2);
public abstract java.lang.Object getValue(java.lang.String param1);
public abstract void removeValue(java.lang.String param1);
(Note,
param1 is the key, or name, of the object being stored;
param2 is the object being stored.)
The application server determines which HTTPSession instance belongs to a particular user by assigning a session id, which is stored in a special cookie in the user's browser. For sessions to work, cookies must be enabled in the browser. Session cookies are not stored persistently, and expire when the browser is closed. Usually, having cookies turned on is not a problem. Many high-volume sites require cookies, including Yahoo and Amazon. There is an alternative in case cookies are turned off on the client browser: to store the session id using the URL rewriting technique. To use URL rewriting, you use the encodeUrl() method of the HTTPResponse interface to append a unique session id to URLs generated by your servlets or JSPs. Regardless of which method of storing session keys is used, the HTTPSession instances are initially held in memory within an application server's JVM. Only the key is held on the client.
Now the question is, how are HTTPSessions stored on the server? The default implementation (which is in the reference implementation of the Servlet JSDK), is to store Session data in memory in the servlet's JVM. This way, it is very efficient to get to an individual HTTPSession instance when it is needed. However, this mechanism becomes a complication when we need to scale our application to handle more users and we begin using more than one server running the same servlet application. To understand the difficulty, look at Figure 1, which illustrates a common setup for a high-traffic Web site.
Figure 1. Load balancing configuration.
In most high-traffic Web sites, the total volume of incoming HTTP requests is too great for a single application server to handle. So, a router (either a hardware router or a software router such as IBM's e-Network dispatcher) is used to divide the incoming HTTP requests among a number of application servers. A routing algorithm, such as a round-robin routing or random routing, chooses which server will handle each particular request. This routing among application servers affects how our application needs to manage session data.
If HTTPSessions are only stored in memory on the server they are created on, that server must receive all succeeding requests from that client. This requirement is called server affinity. This session information will not be available to the servlets running in the other application servers. For some Web sites, server affinity may not pose a problem. However, the way in which routers determine server affinity can pose problems in higher-volume Web sites.
In many routers, a client is "assigned" to a particular application server by examining the IP addresses on the incoming request and always assigning requests from a particular client address to a particular server. However, the reality of today's Internet is that many ISPs have proxy configurations that make it appear to the router that all packets from that ISP are coming from the same IP address. In the worst case, this means that all packets from AOL (which may make up well over 60% of your site's traffic) end up at the same application server. This defeats the purpose of load balancing because one server still ends up with the lion's share of the processing. Also, many corporations now assign outgoing IP addresses randomly, so that two requests from the same client are not guaranteed to have the same IP address. In this case, server affinity cannot be guaranteed.
Server affinity may be acceptable for sites that run on very few machines and also where it is acceptable that users may "lose" their sessions.
Client-Side Solutions
There are two main solutions for storing the session data on the client instead of the server—cookies and hidden fields—each with its own distinct advantages and disadvantages.
Hidden Fields. One of the first mechanisms for preserving session data that emerged was the use of "hidden fields." This option relies on a special feature of HTML to hold session information. The HTML INPUT tag has several different types that allow a Web author to specify how to accept input. For instance, the type "TEXT" results in a text field being displayed on the browser. However, there is one input type that doesn't correspond to any particular UI widget: the type "HIDDEN". Hidden fields have no UI representation, and so cannot be changed by the browser user. The value for a hidden field can be set on a Web page by the application server and can be read back later through the HTTPServletRequest interface's getParameter() method, just like any other HTML field. This is exactly what we need to record session information in the client HTML. A key drawback of using hidden fields to store session data is that we must change the HTML we are writing out to include this new information. The following Java code fragment indicates to do the following:
out.print("");
As simple as hidden fields are to use, they have problems that make them unsuitable for many systems. The first problem is that the JSDK does not provide a way of moving arbitrary objects in and out of hidden fields. In the JSDK, hidden fields are treated in the same way as other HTTP parameters, which is to say they are handled as strings. If you want to use the information in these strings in your programs, you must build a framework for generating the appropriate HTML and parsing out the information again.
Another drawback is that your session information ends up crossing the network multiple times. To understand how this happens, consider the following scenario:
Our wine shopping site has a "frequent buyer program" that has two pages. The first page accepts user information (name, address, etc.), while the second page accepts preference information (Do you prefer Californian or French wines?, Do you like Chardonnays or Merlots?, etc.). The first servlet parses the user information from the first HTML form, and records that information in hidden fields. The second servlet must not only parse out the new HTML form information, but must also reparse out the information from the previous servlet that was rewritten as a hidden field. In this way, each successive page grows and grows, increasing the download time as you proceed farther into your site.
One of the biggest drawbacks of using hidden fields occurs in a heterogeneous site. Many times a new servlet implementation must coexist with legacy CGI and HTML. If you cannot modify these pages, hidden data may be lost as a user traverses between servlets and legacy pages.
Suppose our wine shopping site used to be implemented with CGI programs. Suppose, also that the old shopping cart was implemented as www.winesRus.com/cgi-bin/ shoppingcart. The site currently has all of its HTML pages pointed to this URL. We could change all of these pages to point to www.winesRus.com/servlets/shoppingcart or we could simply alias our servlets with the old link. This alias can be a huge time saver if you have hundreds of catalog pages that point to this link. However, if you are using hidden fields to traverse the site, this solution will not work. Hidden fields must be added to the legacy HTML, and you might as well change the links too.
There are other drawbacks with hidden fields (see the discussion of security in the following section), but they do work well for some purposes. They should be considered in certain sites where security is not high, there is minimal legacy page navigation, and the pages do not "build" on each other excessively—but also where server affinity is not acceptable. Some simple effort spent in building a framework for hidden field generation and parsing can pay back handsomely later. However, as we will see, there are better solutions for most purposes.
Cookies. The next option for storing session data that we must examine is to store the data directly in cookies. As we've already seen in our discussion of HTTPSessions, cookies can also be used to store information on the client browser. They can be used to store not just a client identifier, but the actual session data itself. Cookies have a big advantage.
First, cookies require no HTML rewriting. You convert your session data to a string such as the hidden field example and add it to your HttpServletResponse object like this:
package com.winesrus.tests;
import javax.servlet.http.*;
public class CookieTest {
public CookieTest(HttpServletResponse resp, String state) {
String warning = Please accept this Cookie or bad things will happen to you!"
Cookie cookie = new Cookie("winecookie", state);
cookie.setDomain(".winesrus.com");
cookie.setPath("/");
cookie.setComment(warning);
cookie.setVersion(0);
resp.addCookie(cookie);
}
}
You can retrieve cookies with:
javax.servlet.http.HttpServletRequest.getCookies();
That's it. There is no need to rewrite your HTML pages to get and set cookie data.
The other nice thing about cookies is that you can share your session data with non-Java resources. Your JavaScript and CGI programs can take advantage of this state information because it is passed around with every client request.
However, size limitations can be the Achilles heel of using cookies for session data. A cookie header can store a maximum of 4 K of text. This makes it impractical for storage of large data sets. You also need to be careful of what you decide to store in cookies. This cookie header includes all the cookies from your Web site. If you exceed the maximum cookie size, bad things will happen—for instance, depending on the user's browser, either an old cookie will be lost or your new cookie may not be written. To avoid this, make sure your cookies are not close to the 4 K limit. You should give yourself ample room in case the user has other cookies already in his/her browser from your domain. This entails writing code to check whether the cookie was written successfully. You certainly cannot blindly serialize objects and put them in the cookie—you must be very selective in your marshalling routine.
Another key drawback is that the user can turn off cookies at will. Most modern browsers support cookies. However, there is a minority of Web site users that will want to disable cookies on their browser. This forces you to write either JavaScript in your HTML pages or code in your servlets to detect whether or not cookies are turned on in the user's browser. So, if you use cookies as a session-data storage mechanism, you must always have another mechanism to "fall back" on, or notify your users that the site will not function without cookies.
Yet another drawback of cookies is that there are restrictions on how they are passed around the Web. They cannot be passed to peer domain names.
Let's say WinesRus has bought a new domain name: BeerIsUs.com. We want our users to be able to use the same shopping cart to check out (www.winesRus.com/servlets/ shoppingcart). The problem with using cookies is that the WinesRus.com portion of our site cannot see cookies created by BeerIsUs.com and vice versa. Note, if we had a specific commerce server, called Commerce.WinesRus.com, it could see cookies created at WinesRus.com but WinesRus.com could not see cookies created by Commerce.WinesRus.com.
There are limits on the number of cookies that can exist on a single domain. Some domain name restrictions are browser specific, and you must do comprehensive testing with multiple browsers to make sure your cookies are functioning as designed. Domain name issues can get quite complex using cookies.
Cookies and hidden fields also share another major disadvantage: security. State information stored on the client is insecure. Unless you take the time to encrypt this data, you send all of it back and forth across the Internet as clear text. Even if you do encrypt this data either manually or using SSL, you still do not want to store sensitive business data on the client's machine.
Therefore, the disadvantages of using client-side state may outweigh the advantages of using it as a session-data storage mechanism. A client-side solution may initially seem simple but complexities can multiply quickly. Instead, we need to look at solutions that are initially more complex but that may give better overall results.
Back-End Persistence Approaches
Let's examine the other side of the coin: server-side solutions. Server-side approaches piggyback on client solutions. Use cookies or hidden fields to store the minimum of information that must be carried from request to request—this is normally some sort of global "user id" or key that uniquely identifies the client browser making the requests. Keep the larger set of session data in a third-tier data store that is shared between all of the Web application servers that make up the site configuration. The client-side "user id" can then be used to "look up" the Session data. Many commercial application servers now support a third-tier data store for session management.
A good example of how this works is WebSphere 3.0's shared HttpSession implementation. Shared HttpSessions in WebSphere 3.0 operate like this: Each use of an HttpSession is a transaction onto a third-tier relational database. The transaction begins when the HttpServletRequest.getSession() method is called. The transaction ends either at the end of the servlet's service() method, or when the special method sync() is called on the class that implements HttpSession in WebSphere.
Figure 2 steps through the basic outline of how shared HttpSessions work in WebSphere. Note that this example is to be taken as a high-level description of the roles involved, not as an actual description of the way in which the classes function.
Figure 2. Shared HttpSessions working in WebSphere.
As shown in Figure 2, the process begins with the getSession() method. In this method, WebSphere's implementation classes begin a database transaction on the shared Session database and then retrieve the value of the cookie (named sessionid) used to store the globally unique session id. It then uses this session id to find the appropriate row in the shared session database, and retrieves a long binary or Binary Large Object (BLOB) column from that row. That column contains (in binary form) the serialized HttpSession implementation object that corresponds to the session id. WebSphere then deserializes the HttpSession object and returns it to the requesting servlet.
The servlet can then get and set values into the HTTP session. The client transaction remains open until either the sync() method is called or the servlet's service() method ends. In either case, the HttpSession object is serialized back onto the BLOB column in the database row, and then the transaction is committed.
This same approach can certainly be taken if your application server vendor does not already implement persistent sessions. Your job just becomes more difficult. You can use the JDBC API to store your serialized objects in BLOB format in any major RDBMS vendor's database. While it is a fairly simple process to serialize your objects and throw them in a BLOB, BLOBs have a tendency to perform poorly. (Note, BLOB performance varies widely from database vendor to database vendor, so your mileage may vary.)
A more difficult but better performing solution would be to use JDBC and SQL to store your sessions in non-BLOB fields. This means that you need to map a database schema to your session data. You could not just serialize your objects. Instead, you would need to write your own marshalling routines to flatten and rehydrate your session data.
An even more exotic solution is to use an object-oriented database to cache your session information. This would make the persistence of your objects somewhat easier. You will probably see some performance benefits to this solution. The downsides to the OODMS approach are in database administration, cost, fault tolerance, and the "political" implications of using a technology that is seen as "nonstandard."
Another option is the use of EJB to store the session state. There are two potential ways to do this. First, you could store the key of an Entity EJB in a cookie or rewritten URL, and then use findByPrimaryKey() to locate the appropriate EJB to retrieve the state. This is probably the best overall option because you get scalability on both ends; for example, if your application server does Work Load Management between EJBs, this works well. This option does not necessarily depend on JDBC, because you could use other data sources to store your session data (e.g., CICS). However, complex session data is more problematic because navigating EJB associations can be expensive.
A second option with EJBs is to store the handle of a stateful session EJB in a cookie or hidden field. This would not entail the database query overhead of the previous solution, but unfortunately, most EJB servers do not provide failover capabilities for stateful session EJBs. In that case, you would need to provide your own code to handle failover, which can easily become complicated and bloated.
Conclusion
So, what have we learned? Table 1 details the different options for handling session state that we have evaluated, and the advantages and disadvantages of each. There is no one best solution to this problem. You have to carefully consider the advantages and disadvantages of each solution and choose the one that is best for your particular problem.
| Table 1. Options for handling session state. |
| Solution |
Initial Ease of Use |
Performance |
Session Size |
Security |
Scalability |
Total Development Time |
| In-Memory Session |
Very Easy |
Very Good |
Large |
Good |
Low |
Very Short |
| Hidden Fields |
Easy |
Moderate |
Small |
Bad |
High |
Moderate |
| Cookies |
Easy |
Moderate |
Small |
Bad |
High |
Moderate |
| Vendor BLOB Sessions |
Very Easy |
Moderate to Poor |
Large |
Good |
High |
Very Short |
| Custom BLOB Sessions |
Moderate |
Moderate to Poor |
Large |
Good |
Moderate to High |
Moderate |
| Custom SQL Marshaled Session |
Difficult |
Moderate |
Large |
Good |
Moderate to High |
Very High |
| OODBMS |
Moderate |
Moderate to Good |
Large |
Good |
Moderate |
Moderate |
| EJB |
Moderate |
Moderate to Poor |
Large |
Good |
Very High |
Moderate to High |
Acknowledgments
We would like to thank Patrick LiVecchi of IBM for patiently educating us about the problems of server affinity, and David Williams of IBM WebSphere Services for diving into many of the grimy details of cookies. We would also like to thank Gabe Montero of the IBM WebSphere development team for clearing up issues about session clustering in WebSphere 3.0.
FOOTNOTE
* This assumes your application server implements the Java servlet development kit (JSDK) version 2.0, which most commercial servers (JRun, WebSphere, Weblogics, etc.) do.