Columns

Systems in minutes

• By John D. Williams
• February 1, 2002

A conventional approach to system development won't let us deliver new systems in minutes. It will barely let us deliver them in months. Even with the advent of throwaway Web-based systems, we can't keep up with the demand. Most development methodologies are composed of rigorous processes designed to engineer robust solutions. But methodologies don't help us get systems to the business as rapidly as it needs them, especially when development is on the critical path. For this reason, some IT shops buy systems rather than build them. But experience shows us that it's often a long and costly process to implement a purchased system. Much effort is spent on integration with the existing environment and trying to patch together disparate architectures.

The need for speed requires us to develop and deliver systems differently than we do today. If we can't take the time to build systems from scratch or even buy them, we need to rethink how we create them. Assembly from existing pieces deserves consideration. This has been the promise of components and now Web services—that we can build new systems by assembling existing components and services into new configurations. We aren't there yet, but we are moving in this direction.

But if you assemble new apps out of existing components and services, how do you know which components and services need to be available? You can't use them unless they've already been built, so which ones do you pre-build? The answer is an enterprise architecture.

An enterprise architecture, driven by the business, aligns a business' supporting operations, systems and technology. With an enterprise architecture, we create reference models that capture the alignment. These models define the capabilities of the business, the business' supporting operations and structure, and the relationships between these different aspects of the business. These models are, in turn, driven down to capture supporting systems, their interrelationships and their alignment with the business. This information is further driven down to the level of the technology needed to support the business. This is important because the operations, systems and relationships defined in the models tell you what components and services must be available to support the business. As the needs of the business change, the models are updated and new services and components are identified.

If you simply try to take what you have and promote it as services, you end up with an unaligned hodgepodge of services. Some may be appropriate for assembling new systems, but many won't be. It's like building a house. You can buy the best materials, hire the best contractors and spend enormous amounts of money on a home, but if you don't have a blueprint, you'll end up with a nightmare, not the house of your dreams. An enterprise architecture serves as the blueprint for defining the reusable components and services you need to assemble new systems.

Once you have your blueprint in the form of reference models, you can provision ahead of time the components and services you need. You don't need to build everything from scratch. You might buy or rent new components or services, wrapper legacy systems and present them as services, or you can build new ones. This is where our existing methodologies come back into play. We still want to build high-quality, reliable components and services. Strong engineering methodologies help make that possible. We just don't want to do all the engineering work on the critical path of delivering a new system to the business. With reference models and some planning, we can develop the components and services we need just in time for their use.

Remember, you don't have to provision all possible components and services before you start building systems. Build the services and components you need for a small app and then assemble them as needed. As your base of components and services grows, you will be able to respond more quickly to business requests by simply assembling systems. Some see this vision culminating in the business rapidly assembling its own solutions as needed. The end result—systems in minutes.

But how are these disparate services and components integrated? One of the claims made for assembling systems is that we avoid the cost and complexity of tightly integrated systems. Some claim that in the assembly world, components and services are not integrated; they are federated, or loosely coupled. They use the word integrated to pejoratively describe tightly coupled systems. As a longtime fan of loosely coupling systems, I find this view of integration puzzling. Integration is not a single state; it is a continuum with different dimensions. To better understand the linkages between services, components and legacy systems, let's create a framework to visualize where in the continuum any given integration point would fall.

Points of integration can be simple or complex, and tightly or loosely coupled. When we think about integrating applications, we see that integration can occur at the presentation, business logic or database layers. For our purposes, let's create a framework with two dimensions. Imagine a two-dimensional graph with its origin point in the lower left-hand corner. Let the vertical axis show the type of integration. The type of integration will range from simple to complex. Let the horizontal axis show integration flexibility. We will use the concept of flexibility to encompass a range of connections from tightly coupled to loosely coupled.

When we think of integration type, we can imagine single parameter API calls as an example of a simple type of integration. These may be simple call-and-forget interfaces or they may be call-and-reply interfaces. Integration types become more complex when we pass multiple parameters to an interface. We can also envision even more complex multipoint interfaces. One service may call multiple interfaces of another, or we may have a more complex conversational exchange between components. The most complex interface type is dynamic, where interfaces are discovered and configured when needed. The complexity derives from the challenges of dynamically discovering and configuring interfaces, not just the type of interaction that occurs once the interface is set up.

Likewise, we see a wide range in the flexibility of interfaces. At the lowest level, we have embedded linkages that are hard-coded. This is the old approach that requires components or services to understand an exchange at compile time. When we use something like XML, we move a step up to low-level flexibility. Plain XML lets us define an interface that works with structured content that doesn't need to be defined until runtime. To increase flexibility, we need a common data definition. XML Schema provides this mid-level flexibility.

To reach a high level of flexibility, we need services and components that share a common understanding of their context. With agent technology, this is done using an ontology. We could also use the XML Frames I've proposed to achieve this same goal. The highest level of flexibility is achieved when we can use auto-discovery for dynamic learning and understanding of services and information. This also requires an ontology. Fortunately, we have the beginnings of one in the reference models we create through enterprise architecture.

What does all this have to do with assembling systems in minutes? Loosely coupled services and components are easier to assemble than their tightly coupled counterparts. Simpler interfaces are easier to engineer and use than complex ones. An integration framework can provide guidance and show us the tradeoffs we must make to create the flexible infrastructure, components and services we need in our "systems-in-minutes" world. When we couple this framework with enterprise architecture reference models, we have a solid foundation for a new development and delivery process.