Columns

Advanced Meta Data Architecture

• By David Marco
• August 21, 2001

Corporations are demanding more and more functionality from their IT systems, and meta data repositories are no exception. This article addresses the more complex architectural challenges that arise when implementing a meta data repository that requires more advanced functionality.

While most repository in-itiatives are not attempting to implement these features, it is certainly the type of functionality currently in demand by corporations. It is also important to note that these concepts can be implemented separately or in conjunction with one another.

Typically, the architecture of a meta data repository is one-directional. Meta data sources (such as data modeling tools, extrac-tion, transformation and loading [ETL] tools, front-end tools and so on) flow into the repository and are integrated to satisfy the various needs of the business (business meta data) and IT (technical meta data) departments. A bi-directional meta data architecture allows meta data to be changed in the repository and to then be fed back from the repository to its original source. For example, a user could go through the repository and change the name of an attribute (field) in one of the decision support system’s data marts. This change would then be fed back into the supporting data modeling tool in order to update the physical model for that specific data mart.

This architecture is highly desirable for two key reasons. First, it allows vendor tools to share meta data. This is especially useful in the decision support marketplace. Most organizations that have built a decision support system did so with “best-of-breed” tools, as opposed to integrated toolsets supplied by a single vendor. However, these tools are not integrated with one another and do not communicate easily. Even tools that can be integrated usually require a good deal of resource-intensive manual programming to share meta data. Second, bi-directional meta data is attractive to corporations that want to implement a meta data repository on an enterprise-wide scale. By enterprise-wide meta data I mean companies that want their meta data repository to store information on all of their IT systems (both decision support and operational systems). A bi-directional architecture allows a corporation to make global changes in the meta data repository and to then have those changes sweep through all of the tools in the enterprise.

However, there are three obvious challenges to implementing bi-directional meta data. First, it forces the meta data repository to contain the latest version of the meta data source it will feed back into. Second, it is possible for someone to make a change to the meta data in the repository while someone else is making a change to the same meta data at its source. These situations must not only be systematically trapped, but resolved. Finally, additional sets gram/process interfaces need to be built to tie the meta data repository back to the meta data source.

As we can see, a bi-directional meta data architecture provides a corporation with tremendous value because it allows disparate tools to communicate with one another. This is why the Meta Data Coalition (MDC), with its Open Information Model (OIM) standard, and the Object Management Group (OMG), with its Common Warehouse Metamodel (CWM) standard, have tried to provide an industry-standard meta model. With the creation of an industry-standard meta model, third-party vendor tools can plug their applications into this model and share meta data with each other. Bi-directional meta data will become a reality as a global meta model standard emerges from either of these groups.

It is important to understand that even with a meta model standard, it will take time for the various software vendors to modify their applications to work with the standard. As a result, even after we have a global meta model standard for decision support, and vendors have modified their tools to support it, we will still not have 100% meta data sharing. I believe that it is possible to have a good 75% of the meta data be sharable, which is a much better situation than what we have to deal with today. This will enable these applications to share data, which creates tool interoperability.

Closed-loop meta data
Corporations want to be able to integrate their customer relationship management system (CRM), decision support system and e-business solution with operational systems to provide one integrated business intelligence system. Integrating these systems allows information, especially customer information, to be shared throughout the organization. This enables corporations to provide new and higher levels of customer service. In addition, organizations in service-intensive industries (such as banking, financial services and retail) want to have their systems make certain business decisions for them, without manual intervention.

For example, a retail organization selling consumer electronics might want an e-business sys tem that allows customers to go through the Internet to search for and order whatever components they desire. When a customer selects these components, a program interface would then fire off to the customer relationship management system and other products that the customer has previously ordered could be offered to them. If the customer hasn’t ordered the items in a while, a discount could also be offered. Once the customer has finished shopping, another interface could be sent to the corporate decision support system. This system would then update the customer’s credit rating and return it to the e-business system. This type of integrated, closed-loop system is the mark of excellence.

However, a closed-loop meta data architecture does add some additional complexity to the meta data repository initiative. First, if the meta data that will be fed from the repository to the operational system can also be maintained in the operational system, then the meta data repository must contain the latest version of the operational system’s meta data. Second, it is possible for someone to make a change to the meta data in the repository while someone else is making a change to the oper-ational system. These conflicts must not only be systematically trapped, but also resolved. Third, program/process interfaces need to be built to tie the meta data repository back to operational systems. Few companies are currently attempting this architectural technique; however, it is a natural progression in the architecture of meta data repositories.

Of course this desire for closed-loop systems will impact our meta data repositories directly. A closed-loop meta data architecture allows the repository to feed its meta data back into a corporation’s operational systems. This concept is similar to the bi-directional meta data architecture in that the meta data repository is feeding its information (meta data) into other applications or, in this case, operational systems. The closed-loop meta data architecture is gaining favor in corporations that want to implement a meta data repository on an enterprise-wide level because it allows global changes in the meta data repository to sweep throughout the operational systems of the enterprise. ¹

[This column was adapted from Building and Managing the Meta Data Repository, A Full Lifecycle Guide by David Marco; John Wiley, New York City, 2000. Please see Chapter 7, “Constructing a meta data architecture,” for more detail on this topic and the other more common meta data architectures (centralized and decentralized).]