In-Depth

Battling MOMs

• By Max Dolgicer
• July 6, 2001

There were a number ofmessage-oriented middleware (MOM) products on the market before IBM's release of MQSeries, but IBM's arrival truly validated the market and provided the fuel engine required for growth. Now the world of MOMs is on the verge of further change as Microsoft brings forth its MSMQ product.

Mission-critical enterprise applications are still new territory for Microsoft, and MSMQ does not support a number of features supported by MQSeries, but from a features/functions and robustness point of view, Microsoft's MSMQ clearly can challenge IBM. MSMQ appears less complex to administer, but MQSeries, for its part, supports far more operating system heterogeneity.

The MOM market encompasses many products and vendors. This market includes pioneers like PeerLogic and Momentum Software as well as late comers like Talarian, just to name a few. It includes companies selling products spun out from home-grown message queuing efforts developed on Wall Street, as well as VCOM from Verimation, which arose from automaker Volvo. However, at the end of the day, IBM's MQSeries is the undisputed market leader. Microsoft is trying to change this.

Microsoft's MSMQ is a worthy competitor to IBM's MQSeries middleware if for no other reason than it comes from Microsoft, and the company has substantial distribution channels. In a number of important technical areas, it is even superior to IBM's MQSeries.

Bundled with the OS

Microsoft has released MSMQ as part of the NT 4.0 Service Pack 3. There, it is bundled with the Microsoft Transaction Server (MTS). Microsoft is now beginning to ship this to channels. It is the first company in the middleware space that bundles two premier middleware products with the operating system, thus completely redefining the pricing and statistics models used to indicate and measure the success of any middleware product. Up until now, the success was measured by the number of licenses sold and the revenues derived from these licenses. As Microsoft ships MTS and MSMQ with the NT operating system, it is safe to say that MSMQ will quickly pass MQSeries in terms of the number of licenses sold.

In terms of revenue, the picture is more clouded. IBM's just-released MQSeries V5 pricing model is more complex and expensive than previous MQSeries releases. For its part, Microsoft is bundling MSMQ with two variations of the NT Server, Enterprise Edition (EE) and Standard Edition (SE). In order to compare apples with apples in terms of pricing, an example of a real world application development and deployment needs to be considered. Without going into the details of such an exercise, we predict that a total cost of ownership (TCO) study will show that MSMQ has a pricing advantage over MQSeries.

In technological terms, apples and apples are compared since both products are clear-cut message queuing products. But there are issues to keep in mind. MSMQ is in its first release while MQSeries has been on the market for over five years. Surprisingly, Microsoft's product appears quite complete and stable for the first release. IBM's extensive experience in this market cannot be ignored.

Clearly, while developing MSMQ, Microsoft had an established product to which it could refer. In contrast, with the exception of DECmessageQ (now owned by BEA Systems), IBM had no commercial-strength products on the market to refer to while designing MQSeries. Furthermore, IBM had to labor hard to create a viable market by increasing the awareness of corporate developers and corporate decision-makers about message queuing and its uses. Microsoft does not have to deal with any of these issues. The company, however, has a substantially different problem to deal with.

MSMQ as well as MTS are both positioned as keys to Microsoft's continuous effort to become a provider for enterprise solutions, just like IBM, HP, Sun and others. But it is a big step to make a transition from selling Microsoft Office with endless upgrades, big profit margins, and where little support and consulting are required, to selling software aimed at mission critical applications running banks, airlines and trading floors. Moreover, Microsoft's MOM product is heavily oriented toward its Windows environments. This is key among several issues to consider when comparing middleware approaches.

Software is a very complex and competitive business. That is what makes it so exciting. Good products do not always ensure success, and bad products do not always fail. Of course all of this is more complicated than it appears. There are many items that corporate decision-makers employ before they write checks to software companies and their distributors. However, technical superiority always makes the decision easier. Our partial evaluation shows that both MQSeries and MSMQ are technically sound ready-for-an-enterprise products. This article captures what in our opinion are essential features of message queuing systems.

Heterogeneity

When it comes to supporting heterogeneous operating systems and communication protocols, IBM has the upper hand. This is one of the few features where the verdict is decisive. Message queuing is used to integrate applications that in any modern enterprise are running on heterogeneous operating systems and communication protocols, thus IBM's operating system edge is significant. Microsoft's MSMQ is capable of supporting TCP/IP and IPX/SPX while MQSeries also supports SNA LU6.2. But on the operating systems front, MSMQ runs natively on NT (clients and servers) and Win 95 (clients only), while IBM covers more than 20 operating systems, including strong offerings on OS/390.

But to say that Microsoft has completely neglected support for non-Microsoft operating systems is not true. Largely, Microsoft is relying on third party vendor support. Interoperability with MQSeries is offered via the FalconMQ Bridge available from Level 8 Systems. Using the product, MSMQ applications can communicate with MQSeries applications. Note that the bridge is available on the NT Server Enterprise Edition only and is provided at extra cost. For applications that do not require full message queuing support, MSMQ client software is available from Level 8 Systems as well. Using client software, an application can make calls to a MSMQ Server using the MSMQ API. Level 8 Systems also offers connectivity between CICS Transient Data Queue and MSMQ Queue.

Quality of Service

Quality of Service (QoS) is an umbrella name that encompasses many important middleware features. In its basic form, QoS means support for guaranteed and assured message delivery as well as for message prioritization. Both MSMQ and MQSeries support all three features.

Beyond the basics, QoS encompasses additional features like time-to-live, tracking and notification. Both IBM and Microsoft's products support time-to-live (or what is know in the MOM vocabulary as expiration of messages) where applications can be notified when a particular request has expired and where message queuing systems are capable of handling message expiration. Of course, the sending application should be notified when a particular request has expired. This leads to comment on another requirement -- tracking and notification. Both MQSeries and MSMQ are capable of supporting tracking and notification so that an application can be notified when a request is delivered to a remote queue and the target application receives it. In IBM jargon, the features are called Confirm on Arrival (COA) and Confirm on Delivery (COD), respectively. Using these features, a report message (with or without application data) is generated and can be sent to a specific queue. Of course to make any sense out of this message, an application needs to read the message from such a queue and interpret it according to the application-specific design. MSMQ supports COA and COD features, as well as some additional capabilities. While MQSeries supports only positive acknowledgments for COA and COD, MSMQ supports both positive (ACKs) and negative (NACKs) acknowledgments. A report message is generated when the application message does not make it to a queue or has not been read by a receiving application.

An additional feature under the umbrella of Quality of Service is reporting. In order to track messages within the system, reporting is required. However, for message queuing to detect and report on items such as messages expired or messages reaching the remote queue, it is necessary for the message queuing system to generate and to handle different types of meaningful events. This brings to light another feature called instrumentation.

Broadly speaking, both products are instrumented to support different types of events for different situations. Of course, MSMQ generates NT events that are reported to the operating system log and can be easily interpreted by the NT Performance Monitor. Because MQSeries is platform neutral, these events are logged to different event queues for interpretation by system management tools. The different approaches taken by Microsoft and IBM to system management have a very direct effect on the I/S managers' checkbook.

A good example to demonstrate importance of Quality of Service is a money transfer application. An application at a particular bank wants to transfer money to another bank. The sending application could specify a time-to-live of one hour and also request acknowledgment. A positive acknowledgment could automatically generate a confirmation letter to the customer while a negative acknowledgment could trigger an investigation to find out why money transfer failed. It is safe to say that both products cover QoS quite comprehensively.

Both products support location transparency as far as message queues are concerned. However, there is a big difference in how such location transparency is implemented. MQSeries does not support any built-in naming service where the queue manager, channel and queue names -- and their respective attribute information -- can be stored. Such definitions need to be stored locally. For its part, MSMQ comes with a built-in naming service, called the MSMQ directory, which serves as a comprehensive and distributed repository to store important information. MSMQ directory, also called MSMQ Information Server, contains information about queues, including queue locations, an unique ID associated with queues and queue attributes. MSMQ queues can be created, changed, deleted and located by names either administratively with the easy-to-use MSMQ Explorer or programmatically using the MSMQ API calls. The MSMQ directory, which makes it all possible, is real time and fully replicated, thus preventing a single point of failure.

For now, MSMQ is using it own proprietary directory that also requires a version of SQL Server. MSMQ comes with a license limited for support of MSMQ. Most likely, future versions of MSMQ will be using an NT Server Active Directory that will be the future NT naming service.

Queue order/ selecting messages

Both IBM and Microsoft's products support flexible message ordering and message selection. MQSeries, for example, supports queue ordering based on first-in-first-out and message (0 through 9) priority. Similarly, MSMQ supports queue ordering based on message (1 through 7) priority, and on priority established by the time the message was sent relative to messages with similar priority. In addition to flexible queue ordering, it is important the receiving application will have the ability to receive messages using flexible selection criteria. Using MQSeries, messages based on correlation-ID or message-ID can be retrieved. In the releases preceding MSMQ's commercial release, MSMQ also supported message retrieval based on user defined fields. For now, this feature has disappeared, but probably will reappear in future releases of the product.

When it comes to reading messages, multiple methods should be provided. The most important methods are: (a) where a receiving application waits on a queue until the message (any message or specific message which will, of course, require the receiving application to know the unique message identifier) arrives at the queue; (b) where a receiving application polls the queue until a message (again, this could be any message or specific message) arrives at the queue; or (c) where a receiving application registers a callback, which in turn will get executed when a message arrives at the queue. Basically, the first two methods are synchronous calls while the third one is purely asynchronous and event driven. Unfortunately, MQSeries does not support the third method. MSMQ supports all three methods.

Transaction support

As noted in "Message Queuing Middleware: No Longer High Risk for I/S," (see Application Development Trends, December 1997, p.33), transaction support has three requirements: (a) support for internal transactions where an application can send and receive messages within a single transaction coordinated by the message queuing engine; (b) transaction support coordinated with database updates where an application program can send (or receive) messages to the queue (or from the queue) and update a database in a single transaction without the requirement of a transaction monitor; and (c) support for external transactions, where queuing operations and updates to a database are coordinated by an external Transaction Processing (TP) monitor. As a side point, it is important to know that when applications use transaction support (internal or external), all messages are delivered exactly once, and in the order in which they were sent. This is another case in which both middleware products support the requirements.

The MQSeries V5 comes with an XA compliant transaction coordinator capable of handling updates to databases and message queues in a single transaction. However, only local transactions are supported, meaning that databases and message queues need to be co-located. Not all databases are included, and at this point, only Oracle and DB2 are supported. Support for additional databases will be forthcoming.

MSMQ on the other hand does not come with a database coordinator. It instead relies on Microsoft's Distributed Transaction Coordinator (MS DTC) to handle transactions that involve database and queue updates. From a transactional support point of view, both products appear to be strong. For now, updates to MSMQ queues can only be coordinated with updates to a SQL Server database. This is more of a MTS (or, to be precise, MS DTC) limitation rather than a limitation of MSMQ itself. MTS 2.0 includes support for additional databases such as Oracle, so it is expected that database limitation will be lifted soon.

Language support

The middleware products from both these industry titans support a variety of programming languages. C and C++ support is taken for granted. MQSeries also supports mainframe-oriented languages such as Cobol. MSMQ relies on a third-party vendor for Cobol support. Both products already support Java. As expected there is a big difference between MQSeries and MSMQ when it comes to ActiveX support.

Some complain that one of MQSeries' inherent problems is its limited support for ActiveX controls. This leads to somewhat cumbersome integration with desktop applications (such as Excel) and desktop application development tools (such as Visual Basic and PowerBuilder). As of now, MQSeries supports Automation only (formerly called OLE Automation).

As expected, MSMQ supports a large number of ActiveX interfaces including those that combine multiple MSMQ functions. Developers can use these ActiveX interfaces and if necessary, use the lower level functions of the native MSMQ API. MSMQ also supports dual interfaces, allowing C++ and tools such as Visual Basic and Delphi access ActiveX interfaces. The bottom line here is that MSMQ is a clear winner when it comes to ActiveX support.

Both products' queue managers are multithreaded. The new release of MQSeries addressed the problem of consistent thread support across multiple operating systems. In its previous versions, the queue manager was multithreaded on some platforms (for example, OS/2 and NT) and single threaded on others (for example, Unix flavors). The lack of thread support created some problems, especially for applications that utilize MQSeries in a client/server model. Using a client/server model, clients have no queuing capabilities. They simply connect to a remote queue manager that is responsible for queuing operations. With the queue manager as a single-threaded process, each client's connection created two processes (one in each direction), thus imposing severe scalability limits. In contrast, the MSMQ engine was multithreaded from inception.

Despite the recent advances in MQSeries architecture, there is still a programmatic difference in how threads are treated. The MQSeries API does not recognize threads. The following example demonstrates such a limitation.

Assume that a program has two threads, one (the writer thread) is responsible for writing messages to the queue, and the other (reader thread) is responsible for reading incoming messages from the queue. In the case of MQSeries, before the writer thread can perform any queuing operations, it must first perform two other operations. First, it must connect to the queue manager to obtain the connection handle and second, it must open the queue to obtain the queue handle. Only after the connection handle and queue handle are obtained can a request message be put on a queue, and processed by another application which in turn will put a reply message on the same queue. Naturally since both threads are using the same queue manager and are sharing the same queue, one would expect that both the connection handle and the queue handle can be passed from the writer thread to the reader thread and the reader thread can obtain the reply simply by reading the messages from the queue. Well, what is natural and logical is not always the way things work.

Since MQSeries does not allow these handles to be shared across threads, the reader thread needs to obtain its own connection handle and queue handle simply by repeating operations already performed by the writer thread. MSMQ does not have such a problem since MSMQ fully recognizes threads at its API level and allows context to be shared across threads.

Naming services

A naming service is a must-have component within any distributed computing infrastructure. One would anticipate that a messaging scheme would have a naming service to store information about queues. Unfortunately, MQSeries does not come bundled with any naming service. This has a negative effect upon configuration, administration and monitoring.

MSMQ, on the contrary, comes bundled with a fully functional distributed and replicated naming service (also called Directory Service or MSMQ Information Server) which is one of the reasons why administration and monitoring of MSMQ rates favorably compared to MQSeries.

Administration, etc.

Imagine an application spanning thousands of bank branches with message queues located at each branch. Each branch also has to communicate with the glass house represented by an IBM mainframe. Each branch has request queues and reply queues. Quickly, the enterprise is engulfed with thousands of message queues. To define, administer and monitor these queues becomes a monumental task.

Thus, the ability to define and administer these queues from a central location is an important requirement for a message queuing architecture. In fact, many feel strongly that a system management component should be an integral part of any message queuing system. This view is shared by Microsoft. MSMQ comes with an impressive MSMQ Explorer, a GUI-based console capable of supporting most of the essential functionality for managing MSMQ clients and servers from a central console. This is not the case with MQSeries, although the recent release of MQSeries comes with some Web-based administration. MQSeries does not come bundled with easy-to-use management capabilities; it relies on third-party system management tools, which in addition to being expensive, are difficult to use. People who are buying the MQSeries management product quickly discover the bad news -- buying a MQSeries management product can cost more than MQSeries itself. Alternatively, companies can develop home-grown facilities to meet their requirements. All of this is possible since MQSeries is fully instrumented for management and it provides a Programmable Command Format, which is a management API in canonical format. The bottom line is that MSMQ's built-in management capabilities clearly differentiate it from MQSeries.

Client-to-server/server-to-server

MQSeries is implemented in two modes -- clients and servers. Clients are not capable of performing any queuing operations and are dependent on the message queuing server to send and receive messages. Many companies opt for the client/server mode since it involves much less administrative burden (no queues need be administered on the client) and it is certainly cheaper (client pricing ranges from free to very minimal). Despite such clear advantages, client architecture also has a number of distinct disadvantages.

FIRST MQSeries clients are not supported on all platforms. MQSeries does not support MVS clients. This means that an intermediate tier running an MQSeries server capable of supporting MQSeries clients is required. Using such an architecture, MQSeries clients might, for example, communicate with either NT or Unix-based MQSeries servers that, in turn, provide server-to-server communication to MVS.

SECOND all MQI calls are synchronous. Since MQSeries does not support callbacks, clients can only receive messages by either waiting on the arrival of a message to queue or by continuously polling the queue.

THIRD MQSeries clients exhibit less performance than MQSeries servers.

FOURTH network resource utilization is an issue. The server-to-server communication implies session concentration where only connections between queue managers will have to be established. This is not possible to achieve using client-to-server communications where there is no queue manager on the client side.

Although MSMQ uses different terminology, it can support both the client and server architecture. On the client side it supports something called the dependent and independent client. The MSMQ client is an equivalent of the MQSeries client. As the name suggests, it is dependent on the MSMQ server to send and receive messages. MSMQ also supports independent clients, however, that are capable of supporting queuing operations. This raises a question -- what is the difference between MSMQ independent clients and MSMQ servers? There is a substantial difference.

There is no question that the MSMQ Server is the most important part of the MSMQ product. MSMQ server runs exclusively on the Windows NT Server. In addition to being able to support queuing operations, MSMQ supports API requests from the MSMQ Explorer console application, allowing administrators to manage the system from anywhere on the network. The MSMQ Server includes a routing component that routes messages intelligently between applications and queues across the network.

Often situations will arise where a given message may need to travel complex network routes before it hits the receiving queue. These routes can span fast and slow network links and be part of a LAN or WAN. In addition to having the capability of storing messages in intermediate MSMQ Servers, MSMQ is capable of distinguishing between routes, and routing the message intelligently while taking into consideration a number of criteria such as cost and link speed. MSMQ does not support dynamic routing where the routing decisions are based on run time information available, but it does support static routing where it relies on route information configured by an administrator.

MSMQ provides a home to the MSMQ Information Service, which is a naming service (in other words, a directory) that contains all routing and configuration information for MSMQ. (SQL Server 6.5 is required for this purpose.) This directory is replicated throughout the enterprise, and each MSMQ Server should have an identical copy. MSMQ automatically synchronizes multiple copies of the directory.

To summarize, MSMQ-dependent and -independent clients are both available on Windows 95 and Windows NT. The MSMQ Server is available on NT Server 4.0 only. Something similar to the MSMQ independent client also exists in MQSeries. IBM has released a stripped down version of the queue manager available for Windows 3.1 and Windows 95. This version, called MQWin is capable of supporting queuing operations but does not have all of the intelligence that is built into the full scale queue manager.

Security

Since message queuing technology caters to mission critical applications, security is always a concern. However, the asynchronous nature of the message queuing model poses new problems to security. Most security solutions on the market today, including Kerberos, which is the base for Distributed Computing Environment (DCE) security, make an incorrect assumption that distributed applications are communicating synchronously. For example, the Kerberos mechanism is based on third-party authentication where both principals (sending and receiving application) are communicating with a DCE Security Server (which is assumed to be secure) and granted electronic tickets that are subsequently validated. Since DCE is based on the synchronous remote procedure call (RPC) communication model, there is no danger the electronic tickets will expire before the RPC request is received by the other side. When using message queuing, however, there is no guarantee as to when the request will be received by the other side. Meanwhile the electronic tickets can expire.

MQSeries provides multiple levels of security. The MQSeries API is generally represented by a library that can be protected, thus preventing an unauthorized party from making a MQSeries call. MQSeries also provides security at queue, queue manager and channel levels. For example, it can allow only an authorized party to create a queue, read from a queue and write to a queue. MQSeries also provides security exits at a channel level, adding additional security measures just before the message moves across the network. As part of its message description header MQSeries also supports user-ID, password and alternate password. On MVS, MQSeries is tightly integrated with RACF.

The only security problem not tackled by MQSeries is end-to-end authentication (also called application-to-application security). End-to-end authentication means the receiving application can be assured the message is authentic and from the sending application. To implement end-to-end authentication, messages need to be digitally signed. MQSeries does not provide digital signatures, although a third-party product called MQSecure does. MQSecure is available from Candle Corp., and it implements a RSA encryption.

MSMQ supports end-to-end message authentication. Both message encryption and digitally signed messages are available. Support for such a sophisticated feature in the first release is indeed impressive.

There are other differentiators to consider. MQSeries supports triggers where processes can be started based on the arrival of messages to the queue. Flexible trigger criteria such as queue depth is supported. Moreover, IBM's newly released Version 5 of the product supports large messages (messages sized up to 100 Mbytes), queue groups and coordination between queuing operations and database operations.

For its part, MSMQ can act as a transport for MS RPC. Programmers, of course, inherit all of the advantages provided by RPC including automatic marshaling and the Interface Definition Language, without compromising on features offered by MSMQ such as support for Quality of Service. At this point, RPC calls running over MSMQ transport are not capable of receiving any "out parameters" -- only one-way communication is supported. NT 5.0 will rectify this problem. MSMQ also supports connectivity to Microsoft Exchange. This comes in two flavors. First, two MAPI enabled applications can communicate with each other over MSMQ transport. Second, it supports the Exchange Connector where Exchange users can send messages and forms to MSMQ applications for processing. To be fair, it should be noted that MQSeries has been well integrated with Lotus Notes where it provides important connectivity between Lotus Notes Servers and non-Notes applications. It also provides support for MQLSX which allows the MQSeries API to be called from LSX, the Lotus Notes scripting language.

Since MQSeries has been around longer than MSMQ, it provides support for Windows 3.1, thus supporting the huge installed base of Windows 3.1. It remains to be seen whether Microsoft will address this issue. For now, MSMQ clients (dependent and independent) are only available on 32-bit operating systems (Windows 95 and Windows NT).

Rivals show sense

Despite all the rivalry between Microsoft and IBM in the last years, common sense sometimes prevails. Both companies recognized that interoperability between the two products is important and the MSMQ-MQSeries bridge, called the FalconMQ Bridge, is available. This means that an application using the MQSeries API is capable of communicating with an MSMQ based application. As good as this solution sounds, a number of issues remain unresolved. These include, for example, administration, configuration, monitoring, and security questions.

Stay tuned ...

There is no question that MQSeries is the undisputed leader in this important and rapidly growing MOM market. The experience and credibility that IBM has in providing solutions for mission-critical applications cannot be ignored. Despite some shortcomings, MQSeries will continue to be the market leader. Still, the lack of built-in system management capabilities, which increase the cost of ownership for the product, the lack of a built-in naming service, the limited security model and most importantly, the product's complexity might eventually slow down IBM's market penetration and may even compromise its number one position.

While MSMQ (and Microsoft as a whole) still lacks real world experience in development of mission critical applications, it clearly offers a strong alternative from a features/functions and robustness point of view. It does not support all features supported by MQSeries, but it does offer support for message encryption and digital signatures, a built-in comprehensive directory service and, most noticeably, bundled easy-to-use administration, and a few others.

Will MSMQ really challenge MQSeries hegemony? What will be the impact of Microsoft's entrance into the MOM market? It is too early to say. One thing is clear, companies looking for a full implementation of message queuing middleware across many operating systems will quickly find out that MSMQ's lack of platform support will make the decision to go with Microsoft difficult. Customers who are ready to endorse NT as their primary server operating system either for a specific set of applications or for the entire enterprise might find MSMQ appealing even though their mainframes (with, possibly, MQSeries) are there to stay. Lack of heterogeneous operating systems support for MSMQ and the complexity of MQSeries might ultimately create an opportunity for a newcomer.