Transaction processing for the masses -- ADTmag

Transaction processing for the masses

By Max P. Grasso
March 1, 2000

Every Windows desktop or server running Windows 98/ NT/2000 is inherently a transaction-capable machine. Microsoft, after having made a huge effort to lower the barrier to entry into transaction computing, first with the Open Database Connectivity (ODBC) functionality and then with the Microsoft Transaction Server (MTS), has made transaction services part of its core platform. If you install Windows 2000, you no longer need to install MTS with a separate package - it comes with the OS. It comes together with a bag full of other computing goodies, like queuing and asynchronous remote invocations, ready to support any enterprise system - an online commerce system, a Web-enabled customer relationship management (CRM) system, or a supply-chain automation system based on transactional workflow.

Yes, out of the box, your Windows machine can participate with transactions that may span local and remote databases, as well as send and read messages within atomic units of work.

Of course, distributed access and manipulation of data is not all there is to transaction processing. Issues concerning scaling to a large number of concurrent activities, as well as the control and management of resources, are also extremely important for transaction processing. And your Windows system provides state-of-the-art solutions for all these issues. Solutions that a decade ago were only available on mainframes were very expensive and much more primitive.

This article presents the features of the powerful transaction processing functionality available on Windows — functionality that is still identified by the MTS moniker, but that in less than a year will blend completely with the Windows 2000 component services: COM+.

The first modern app server

A short description of MTS can be the list of the main services it provides: object monitoring, transaction services and security services. If these services strike you as the core functionality of a modern application server, you are not off the mark. Indeed, MTS is what makes Windows an excellent application server platform. And if one compares the MTS architecture with the later Enterprise JavaBeans (EJB) specification, one can see how Java vendors have reused its seminal architecture.

Given all this, it makes sense to look at MTS while keeping an eye on the framework presented in a past article on application servers ["Application servers unmasked," Max P. Grasso, Bharat Gogia and Hoa Nguyen, Application Development Trends, October 1999, p. 69]. In that article, the functionality of application servers was divided into a number of categories:

1. Presentation services 2. Distributed object services 3. Transaction services (including object monitoring and transaction management) 4. Security services 5. Integration services 6. Deployment services MTS mostly covers categories 2 to 4, letting other Windows technologies and Microsoft products complete the line-up. (See "DNA in a nutshell," p. 52, for a quick overview of these other technologies.)

MTS builds upon Microsoft's Component Object Model (COM) and its communication-enabled version, Distributed COM (DCOM). MTS, using the COM facilities, defines a set of interfaces and protocols that components can use to their own advantage. Components that use the MTS-defined interfaces according to the specified protocols can enjoy a number of valuable services, allowing the construction of highly efficient and scalable systems. These components are often called MTS components. MTS also defines an environment for specialized system components, called resource dispensers, which can, via MTS, provide services to other components in the most efficient fashion.

Wrappering objects - MTS works much of its magic by completely isolating the MTS components in an execution sandbox, intercepting all communication between the components and third parties. Whenever the components create an externally visible object, MTS puts an object of its own, which we will call Context Wrapper, in between the component's object and the rest of the world. The Context Wrapper externalizes interfaces identical to the interfaces in the original object. Its function is to intercept the requests for the object and to call the MTS runtime (often referred to as "MTS Executive") to act upon the security, transactional, threading and object life-cycle conditions. After MTS has done its processing, and the conditions for execution are positive, the request is relayed to the actual object.

A client holds what looks like a reference to the component's object, but the real object may not even exist. It may be created by MTS when a method invocation comes in and, if it does not contain state, it may eventually be destroyed to save on resources. Components are aware of these life-cycle events because, according to the protocol, MTS invokes the appropriate methods of the object's "IObjectControl" interface (see Table 1).

MTS also uses the interception mechanism to deal with the issue of concurrency control. While concurrency is an important element of scalability, it is probably the single most likely cause of software defects. At the heart of the matter is that concurrency introduces nondeterministic behavior, so that software defects may only appear when specific timing or stress conditions are met - providing no symptoms during testing and debugging, and bringing the system down at random in production. Combine with this the fact that only a minority of the programming population can handle concurrency in a disciplined fashion, and you can imagine the potential for disaster.

Controlling concurrency - Since it is not possible to ban concurrency altogether in this proud distributed computing world, MTS takes the tack of providing a controlled form of concurrency, introducing the concept of activity. Whenever a client first tries to access an object from a component, an activity is associated with the client and a Single Threaded Apartment (STA) out of a pool is associated with the activity. Requests from the client are all redirected to the STA message queue and picked up from processing as soon as the STA's thread has completed processing a previous message. Processing requests in this way ensures that the method logic is executed at the object level in an orderly, serial fashion.

Eventually though, it became obvious that, for performance reasons, some components are better off implementing their own synchronization logic despite the dangers. In Windows 2000, concurrency control can be set at different levels for each component, allowing high-performance, carefully tuned components to execute unhindered by external thread control.

Object pooling - A large-scale system is likely to respond to many clients, and each client will create and release objects from components within the system. The more expensive the objects are to create, the more beneficial it is to reuse them across more than one client activity. To be reused, objects must be returned to a pool rather than destroyed. And they must only be created when no objects are available to be taken out of the pool, because all of the existing objects in the pool have already been taken out and are busy.

While MTS under Windows NT 4.0 gave only a hint of future pooling capabilities, the new transaction handling functionality in Windows 2000 includes a full-fledged pooling mechanism. A number of parameters can be set through the administration interface, including the minimum and maximum size of the pool.

Resource pooling - While pooling for component objects is recent functionality, MTS has always supported the reuse and pooling of expensive, transaction-sensitive system resources.

Database connections are a typical example of such resources. In most component-based systems, objects access one or more databases for the persistent storage of client- or business-related state. To do that, they need to create or somehow get hold of database connections. Right now, database connections are notoriously expensive and slow to create, so having objects create them as needed is unthinkable.

MTS defines a general architecture for the pooling and reuse of objects that are expensive to create, using the concept of resource dispensers. For the all-important case of database connections, MTS provides a concrete implementation of a resource dispenser, the widely used connection pool. As its name suggests, the connection pool manages a pool of connections and, upon request, searches it for connections that can be reused. The search has a preference for connections that are already enlisted in the appropriate transaction (connections are obviously sensitive to the transaction context); if none are available, however, any appropriate connection can be taken from the pool. In the rare event that no fitting connection is found, a new one is created.

Because the necessity for pooling database connections is so pervasive, connection pooling technology was eventually bundled with the Open Database Connectivity (ODBC) services.

MTS also provides a special resource dispenser called the Shared Property Manager for the efficient sharing of memory and locks, in the form of properties and property groups, among objects within a package.

Finally, another widely used resource dispenser is Microsoft's OLE DB dispenser, which works with connections to OLE DB data providers. (OLE DB is Microsoft's architecture for uniform access to all data sources, including relational database systems.)

Naturally, connections to other resource managers, like queuing services or object databases, would also greatly benefit from a custom resource dispenser - and a vendor of any such product should consider bundling an appropriate dispenser with it.

Transaction demarcation

By their nature, components are not built with a single, defined use scenario in mind, but rather are built to satisfy a specific functionality need across as many scenarios as possible. This is so that the component can be reused as often as possible.

However, the demarcation of transactions tends to be closely tied to a specific scenario, encompassing the actions of many components into one atomic unit of work. The programmatic starting and ending of a transaction within a component would make the component very specific to the scenario at hand and all but destroy its potential for reuse.

By supporting the declarative demarcation of transactions, MTS provides a solution to this problem by allowing the programming of components free of the chore of starting and stopping transactions. Rather, the component will need to specify its transaction needs to MTS. If the component uses a database and requires transactional atomicity, the transaction support for the component is set to "Required." Once this is done, it is up to the MTS Context Wrapper to intercept all method invocations, check that they are within a transaction, and, if this is not the case, start a new one.

Transaction coordination - If you need to reach multiple databases and update them atomically, you need to execute a distributed transaction. For example, if you withdraw money from an account in local database A to deposit into an account in remote database B, you want the two operations to be executed atomically. This means that you either want both operations executed or neither operation executed. Otherwise someone will lose money.

One common way to achieve this atomicity is by using a distributed transaction coordinator. MTS comes with something of the sort: the Microsoft Distributed Transaction Coordinator (MS DTC). Because it is essential for the correct execution of distributed transactions, the MS DTC was also bundled with the SQL Server. It is now part of Windows 2000, together with all of the MTS functionality.

The coordination of a distributed transaction relies on the cooperation of the MS DTC services running on each one of the machines where resource managers have been touched by the transaction. The coordination of a distributed transaction typically consists of two rounds of message exchanges (hence the name "two-phase commit protocol").

The coordination is completed when all the resource managers - the keepers of persistent data - that have been touched by transactions are aware of the transaction outcome. If the transaction was committed, its data changes need to be made "durable," otherwise they need to be undone cleanly.

Entities that do not hold persistent data (and therefore are not resource managers), do not care about participating in the transaction coordination process, though they will be interested in the transaction outcome. This is the case for all but a very small number of MTS components out there.

Because coordination is opaque to all of the elements in the system, with the exception of the DBMSs and the MS DTCs involved, readers should refer to the excellent book Transaction Processing: Concepts and Techniques by Jim Gray and Andreas Reuter (Morgan Kaufmann Publishers, 1993).

Security services

The Windows platform provides system-level support for Access Control Lists (ACLs). These are lists that define per object access permissions and contain access records for groups or users. While ACLs controlling single objects or aggregates of objects seem like a very flexible mechanism, they are not the best solution outside the system realm because the correct mapping of access policies into actual ACL configurations increases in complexity with the number of ACLs around.

For an example of this, just look at how often administrators have to change file ACLs to accommodate the access needs of the users in your company - and how often access security holes are created by such processes.

Moreover, business functionality tends to model access the same way business processes do: by role. In real life, for instance, human resource personnel can view everyone's salary but cannot modify the information, while an employee has the right to view his or her own company file, etc. It is therefore more natural to define the "human resource" role and let the business functionality check for it, rather than to make sure we actually provide the appropriate access by setting abstract read/write permission on some internal object.

In short, though, technically speaking, one can convert roles into ACLs and vice versa, roles encourage the use of higher level privilege attributes, rather than lower-level object permissions. Since MTS is geared toward the support of business functionality, it rightfully eschews the use of ACLs in favor of the more understandable and manageable roles.

Roles - Roles can be described as groups that exist only within the confines of a specific application and are associated with an application-specific set of privileges. Roles having the same name - Administrator or User, for example - within two given apps have nothing in common other than this name. Because roles for different apps have nothing in common, they do not share their members either.

MTS uses component packages as the deployment unit of functionality (application), so MTS roles exist within the boundaries of component packages. MTS provides full support for the administering of role memberships, as well as for the administration of role access to components.

Indeed, a very granular declarative access control mechanism is available. Roles can be allowed to access specific components, specific interfaces and, in Windows 2000, under specific methods within an interface.

As one can easily imagine, MTS implements access control by intercepting all method invocations to an object and checking whether the caller is in any of the roles with permission to execute the call. If the caller should not be allowed to execute the method, an error is returned; otherwise, the call is passed on to its destination.

In some rare cases, the granularity of the role-based access control is not enough. It is still very easy to check programmatically whether the caller is in a role and to take action accordingly.

Using the MTS security facilities, components can be built without consideration for the users and group(s) present within the administrative domain. Rather, when the component or the package of components is being built, a number of roles can be defined and named. For each component, interface or method, one can specify the roles required for execution. Without writing one line of code, access to the business functionality is scrutinized by MTS. Both the role membership of users and groups, and the execution permission given to each role, can be modified using an administrative interface (the MTS Explorer or, in Windows 2000, the Component Services management console). This will allow components that rely on the MTS declarative security to better adapt to different deployment situations without requiring code changes.

Component deployment and load balancing

The functional unit used by MTS for administration, deployment and execution time is the component package. Packages can be created and configured using the Package Wizard (renamed COM Application Wizard in Windows 2000). Once a package is created, components can be added by using the Component Wizard to import them from Dynamic Link Library (DLL) or by selecting them from among the ones that are already registered.

This process is very flexible, as components can be grouped within packages in many ways. It is important to consider, though, that components within a package share the execution process, and therefore the identity under which they execute; they share a more direct though less-secured access to each other and they also share properties maintained by the Shared Property Manager. All of this can lead to faster execution if the components trust each other and are built to take advantage of such intimacy. On the other hand, components that are not built to cooperate may stop working altogether if they are included in the same package.

MTS is also capable of exporting packages from one machine to another, thus facilitating the build, stage and deploy process.

Currently no load balancing is available for MTS components. The need for load balancing is proven by the many simple load-balancing schemes included in the application code of MTS-based systems. In 1999, it seemed that Microsoft had responded to this need when preliminary load-balancing functionality was delivered with one of the beta releases of Windows 2000. Soon after, Microsoft pulled the functionality out, and is now scheduled to release it separately by the end of this year.

I trust that most crafty MTS developers will keep using home-brewed versions of load balancing for some time to come.

MTS has brought about transaction computing for the masses, while raising Windows way above its desktop computing origins. I have detailed all of the most important features of MTS, the ones that, in my experience, have the most impact and the most use in real-life projects. Throughout, it transpires that the MTS functionality in Windows 2000 makes a better MTS. Irritating limitations have been eliminated and added functional elements, like object pooling, allow faster performance and better scalability. All in all, the major benefits come from the fact that, in Windows 2000, MTS no longer really exists. Rather, the new COM+ moniker gives a name to the unified and extended services of COM, DCOM, MTS and MSMQ.

This unification of services, as well as the synergy among them and with other Windows 2000 services like Active Directory and Biztalk, makes the Windows 2000 platform a very strong contender as a platform for all forms of e-business solutions.