In-Depth
Book excerpt: Architectures for integration, extension
- By Luke Hohmann
- May 1, 2003
This article is excerpted from Chapter 8 of “Beyond Software Architecture: Creating and Sustaining Winning Solutions” by Luke Hohmann. Used with the permission of the author and Addison-Wesley.
 Integration is the degree to which your system can or must work with other systems, usually programatically, in order to produce the expected product. Extension refers to the degree with which your system can be extended to produce an augmented product. This article will discuss the motivations for creating architectures that can be integrated and extended with relative ease and the business ramifications of doing so. |
Customer control — The driving force
The motivations for integration and extension are similar: In complex systems, both provide the ability to create a superior product. In some circumstances, integration and extension shift work from you to your customer. Paradoxically, this can increase customer satisfaction, much like our satisfaction with an ATM machine, even though we’re assuming responsibilities formerly taken by tellers. Part of the reason is that we’re in control, which most of us like even when it means more work.
Some of the strongest motivations for creating systems that can be integrated and/or extended with other systems include the ones discussed in the following sections.
You can’t predict, but you can plan: In today’s complex environment it is impossible to predict precisely how your customer will want your system to behave in every context. By providing customers with one or more ways to integrate or extend the system, you won’t have predicted the future but you will have planned for it. Plug-in architectures, such as those in Adobe Photoshop or Web browsers, allow components to be added in well-defined ways. They are an excellent example of adding functionality that enhances an existing system.
Customers hate to hear “We can’t do that”: When you can’t extend a system, you can’t respond to customer needs. Ultimately, this means you’ll have to eventually say no to a customer request, which, not surprisingly, customers hate to hear. A good strategy for integrating and/or extending your system will help you convert “No” to “Yes, we can do that. It might take a bit of work, but here’s what we propose.”
The larger solution comprises multiple smaller solutions: Many enterprise-class business systems are not designed to work in isolation. Instead, they work with a supply chain to solve a larger problem. One example is a Web storefront or online ordering system, which cannot work without integration. In such implementations several systems must be integrated to produce the expected product, including a catalog system, a storefront engine, a content engine, a payment processing engine, and backend order tracking and accounting systems. If you’ve participated in the creation of any of these systems, you already know that an easily integrated architecture is essential to success.
You want information not in your own system: There are times when the only way to produce an interesting report or analysis is to integrate the data contained in one system with the data contained in another. A common example of this is when you integrate clickstream data from Web servers with customer purchase transactions.
You want to increase switching costs: Switching costs refer to what your customer will pay should they stop using your system and start using another. These costs include a number of things, such as re-integrating your solution into a competitor’s. Extension and integration options don’t increase switching costs until they are used by your customer. At that point, switching costs dramatically increase because your customer not only has to replace your system but also has to replace all of the integrations and extensions they created on top of it. The more proprietary the extension and integration options, the greater the cost to switch.
You want to create a product ecosystem: Regardless of your company’s success in creating, marketing, and selling your product, you can often achieve greater benefits by sharing your success with other companies. I think of this as creating a product ecosystem, in which various entities interoperate to achieve one or more mutually beneficial goals. A classic example of an ecosystem is the set of companies that create plug-ins for Adobe Photoshop. These plug-ins extend the product in well-defined ways, usually providing specific functions required by niche markets. Key participants in this ecosystem benefit, including Adobe, the company creating the plug-in, and, most important, the customer.
It’s common sense: A good ‘tarchitect’ [a term I use to refer to the traditional software architect or chief technologist] will just about always add some form of API to his system to make it easier to test via a solid automated regression test framework. A good ‘marketect’ [that is, the product marketing manager, business manager, or program manager responsible for the system] will seek to leverage this technical investment -- after all, who knows what the future may bring, and all that may be needed to add this feature to the product is some documentation.
Regardless of your motivation, it is essential that the development effort embrace the notion of an API as a commitment to your customer. A customer, partner, or other entity, such as a system integrator or value added reseller (VAR) that chooses your APIs is tightly bound to your product and your company. This commitment is for the long haul.
Layered business architectures: Logical structures
In one of the most common system architectures for business applications, subsystems are logically and physically arranged in layers. These architectures, which form the foundation for frameworks such as J2EE, provide an excellent case study of integration and extension. In this section I will briefly review layering the architecture. In the next, I will discuss extending the architecture.
The user interface layer: The user interface layer presents information to the user and manages the user’s interactions with this information. It is usually graphical -- either as a heavy client that runs as an application, or a thin or light client that runs in a browser. Other increasingly common forms include voice and handheld. The user interface layer often must shoulder the bulk of the work in internationalized applications, which are always more difficult than they seem. Some of the worst user interface problems I’ve had to resolve dealt with internationalization.
The most essential thing to keep in mind when constructing the user interface is that it not contain any application or business logic. This kind of logic belongs in other layers. In thinking about where to place application logic, ask yourself the following question: “What parts of my system would have to change if I replaced the current user interface with an entirely new one, such as replacing a geographical user interface with an automated voice response system?” If the answer includes changing substantial portions of your application code (e.g., edit or validation checking), chances are good that your user interface layer contains logic that should be in the services or domain layer.
Many enterprise applications split the user interface between two layers -- one that deals with the actual presentation of information and one that mediates the “micro workflow” between a given user interface and the services layer. For example, suppose you’re creating an application to manage flight reservations. If the specific user interface is a browser, you may be able to acquire all of the necessary data in one screen. If it is a phone, you may need to coordinate multiple dialogs. Because these operations are based on a specific kind of user interface, they belong in the user interface layer. The “micro workflow” layer may be responsible for reformatting any domain-specific data for the interface.
I strongly recommend a simple “command-line” interface, which is easy and fast for developers to use, facilitates many forms of automated testing, and is trivially scriptable using a language such as Tcl or Perl. It can also be easily implemented on top of the other layer’s model via a simple API.
The services layer: The services layer provides various application-defined services to the user interface and other applications. These services may be simple, such as obtaining the current system date and time, or complex, such as changing or canceling a flight reservation in an airline reservation system. Complex services are often implemented as transactions, with complete transactional semantics (e.g., rollback). Thinking in terms of services is one of the most important steps if you’re thinking about exposing some or all of your application functionality in a Web service.
There can be a close correlation between services and use cases. At times, a whole use case may be represented as a single service, at other times, individual steps within one may be. CRUD operations (create, reference, update, and delete) are often represented as services as well.
The domain model layer: The domain model or domain layer represents the fundamental business concepts and rules of your application domain. I consider it an optional layer in enterprise applications, only required when business rules are too complex to be represented in simple services or when object structures are more efficiently represented by in-memory representations.
When the domain model is needed, it often emerges as the “core” of the application. In other words, instead of thinking of your architecture in layers, think of it as an onion. The center is the domain model, and other layers are built, or grown, depending on your development method, around it. It needs to be correct.
I admit, this visualization suffers a bit because it doesn’t consider persistent data. More dangerously, it could imply that the domain model is more important than the persistent data model, when in fact in most applications these two elements of your tarchitecture are equal. [The ‘tarchitecture’ is the dominant frame of reference when developers think of a system’s architecture.] However, the onion analogy reinforces that the domain model is the core of a good application.
Decoupling the domain layer from the user interface and transaction management layers provides substantial flexibility in system development. We might replace the screen presented to a service agent with an interactive voice response system or a Web page without changing the underlying application logic (provided they have reasonable interfaces and appropriate service objects to make this replacement -- more on this later). It also contributes to cohesion: Each layer of the architecture is responsible for a specific set of related operations.
I do not mean to imply that the domain model must be constructed before the user interface is designed. While it is often helpful to design the user interface after the preliminary domain model, I have worked on several successful projects in which the user interface was prototyped first, using paper-and-pencil (“lo-fidelity”) techniques. Once the user model was validated, the development of the domain model was relatively straightforward. During implementation the domain model was implemented first, and the user interface followed quickly thereafter based on the previously agreed-to public interface the domain model provided.
The persistent data layer: Most business applications rely on a database management system to manage the persistent storage of objects. In enterprise applications, the most common approach is to use a relational database and to create a separate layer to manage the mapping between objects or service buyers within the domain and objects within the relational database.
This is not as easy as it may sound. Complex object structures, objects comprising data from multiple sources, objects with complex security restrictions (such as operations that can only be performed by certain classes of users), or transactions that involve large numbers of objects all contribute to the challenge of efficiently mapping domain objects to relational databases.
For these reasons, there are times when it makes sense to structure the domain model so that it can work easily and efficiently with the underlying database schema. Indeed, if the schema is unusually complex, or if the performance requirements are particularly severe, it may make sense to forego the domain model entirely and simply connect the services layer to the schema. This may seem counter-intuitive, especially if you’ve been trained in object-based design methods. However, the reality of many enterprise applications is that creating a rich domain model and then relying on an object-to-relational mapping to store and retrieve objects just isn’t worth it. A better approach is to define an appropriate set of services that connect to the database through SQL statements and the occasional stored procedure and/or trigger.
Another interesting way that you can relax the formal structure of a layered architecture is by moving business logic into the database. Architectural purists will tell you that this isn’t a good thing, and they’re right. Moving business logic into the database usually means writing stored procedures and/or triggers, which aren’t portable. It also can mean that the persistent data layer team is not using SQL effectively.
Still, there are times when it is practically appropriate to carefully move business logic into the database. The first concerns performance. If your design involves iterating over a bunch of records, you’re wasting valuable resources by moving data from the database and into another tier. Such logic, especially for very large databases, may be better off in the database. A second motivation is when you’re working with data that has very sophisticated constraints, such as when you want to conditionally delete records. A final motivation is when you want to absolutely, positively guarantee that one or more actions are taken no matter how data is manipulated. This is especially important when you allow integration of your system at the database layer. By definition, this approach bypasses whatever business logic has been built into the services or domain layers, making such integrations riskier. By moving critical business logic into the database, you can always make certain it is executed.
Variations on a theme: As a generic architecture, [a layered system architecture] is a starting point for the design of loosely coupled, highly cohesive systems with the flexibility to handle complex problems. Of course, it is a simplification and abstraction: Distributed computing, legacy systems integration, and/or structural relationships or choices made to support specialized databases or hardware can change the picture.
For more information about this book, please go to the Addison-Wesley Web site at www.awprofessional.com