In-Depth

Ten Rules for Building An Intelligent E-business

• By Wayne W. Eckerson
• January 1, 2001

E-business is big business. It is estimated that more than $377 billion worth of goods and services will be purchased over the Internet this year. Companies are reshaping core business strategies to exploit this new e-channel. But which companies are poised to succeed and which are destined to fail?

The key to e-business success depends largely on a company's ability to capture, analyze, leverage and distribute information. Today, the information a company produces is often more valuable than the products it sells. "The key to wealth," writes Thomas Friedman in his book The Lexus and the Olive Tree [New York: Anchor Books, 2000], "is how your country or company amasses, shares and harvests knowledge."

In 1998, top e-businesses figured out how to design attractive and functional Web sites, manage electronic catalogs and process credit-card orders. In 1999, leading e-businesses discovered how to integrate their Web sites with back-end order, fulfillment and shipping systems. In 2000 and beyond, the hallmark of a successful e-business will be its ability to understand and proactively cater to customers on an individual basis, and to link internal systems with those of its distributors, suppliers and third-party exchanges.

Having mastered the operational elements of running an e-business, successful companies will shift their focus to adding intelligence to their e-businesses. These firms will integrate e-commerce and business intelligence capabilities to optimize customer interactions through both online and offline channels, and to speed the flow of information and products across an extended supply chain.

Intelligent businesses establish informational and organizational processes that reinforce a company's primary strategic objectives and address key performance metrics. Invariably, these companies leverage the Web and other touchpoints to increase customer satisfaction, loyalty, profitability and lifetime value at the expense of their primary competitors. They also improve supply-chain efficiencies by giving distributors and suppliers greater visibility into internal systems so they can replenish stock proactively or fulfill online custom-configured orders directly.

Intelligent businesses also understand what makes the "e-channel" unique and build data capture and delivery systems to exploit this new medium. These companies know that e-business is part of a larger customer management strategy that calls for creating and leveraging robust customer profiles required to drive a multiplicity of customer-facing applications.

Impact on data warehousing
Building an intelligent business for the e-world is not easy. There are many cultural, organizational and technical issues to address. As data warehousing and business intelligence professionals, you will witness many changes. In fact, you will be responsible for establishing many of the new information "stores" and "pathways" required to support new e-business processes.

Ultimately, this means that the form and function of the infrastructure you now manage will need to change in fundamental ways. Traditional data warehouses—if there ever was such a thing—will need to give way to more customer-centric, operationalized and distributed Web architectures that facilitate high-performance access by end users and applications across organizational boundaries. These architectures will be designed with the "customer" in mind so that analysis refines rules that inform touchpoint applications how to best service and address the individual customer's needs and wants.

Those of you who cling to old notions of providing information access and delivery will be swept aside in favor of forward-thinking individuals who understand the dynamics and requirements of the intelligent e-business.

10 rules of the road
Below are 10 rules for building an intelligent business for the e-world. These rules are designed to help you understand the new role you need to play within your organization and the changes you need to make to your company's data warehousing infrastructure. The more proactive you are in anticipating and addressing e-business requirements, the more successful you and your team will be.

(Note: The following 10 rules assume that your company has deployed a data warehousing infrastructure that is more than just a glorified operational reporting system. Business users should be using analytical tools running against a data warehouse to make tactical and strategic decisions that ultimately affect the bottom line of the company. If this is not the case, then you still have to "sell" upper management on the strategic importance of business intelligence technologies.)

Rule 1: Integrate
The first task of any intelligent business is to integrate the "e-channel" with the rest of the company. One of the biggest problems with e-businesses today is that they are run as silos of processing activity without connection to the rest of the company. This results in two major problems.

First, it is one thing to build a Web site and another to link it to back-end order-entry, fulfillment and shipping systems, as well as suppliers' systems. Without tight integration, companies may list products for sale on their Web sites that they no longer have in stock or cannot ship by a promised delivery date. Many prominent e-businesses learned this lesson the hard way during the 1999 Christmas season.

Second, although e-commerce engines promise to "personalize" a Web site, there is not much "person" in their personalization strategies. At best, these engines may personalize a Web page based on an individual's IP address, referring site or current page view. This is fine as it goes, but it misses a huge opportunity for companies to deliver compelling value to customers by personalizing their interactions based on a comprehensive customer profile kept updated in real time.

Challenge: Data warehousing managers hold the keys to the company's information repository. Although there are many ways to build an integrated, streamlined supply chain and customer intelligence capabilities, each can—or should—involve data warehousing principles. In particular, data warehousing managers should be involved in creating comprehensive, real-time profiles of customers and inventory/shipping configurations.

Rule 2: Inflate
One of the unique characteristics of the e-channel is that we can collect an unprecedented amount of information about customer behavior. Not since the days when we all shopped at the corner store have we been able to create a fully dimensional view of an individual. We can record each customer's every move and thought. We know where they came from, what they are looking for, what they browse, what they put into their shopping cart and what they take out, what they purchase and for whom, and where they go when they leave.

Not surprisingly, this detailed customer information is voluminous—and intelligent e-businesses want to capture most of it. For example, one global company captures 2Gb of clickstream data each night from two dozen Web sites worldwide and loads it into a data warehouse for analysis the next day. And this company has yet to correlate this traffic data with transaction data or customer behavior gleaned from other touchpoint applications. Clickstream data warehouses of hundreds of terabytes will not be an uncommon occurrence.

Challenge: Today, most companies either fail to capture valuable clickstream data, or they maintain a limited history because of the data volumes. Systems managers need to enhance the scalability of their data warehousing platforms so that they house much larger data volumes without sacrificing query or reporting performance. They also need to parallelize and streamline ETL processes to meet increasingly tight batch-load windows. This may involve devising ways to compress and abbreviate clickstream data to reduce its volume before it ever gets loaded into the data warehouse, or trickling updates to the warehouse on a continuous basis.

Rule 3: Differentiate
In an intelligent bus-iness, a data warehouse becomes differentiated by the various roles it must play. In most cases, companies will create distinct systems and data structures to support these roles.

For example, intelligent businesses require companies to "operationalize" their data warehouses. Customer-facing applications—such as the Web, call centers and retail outlets—leverage historical customer profiles and analytical rules in real time to generate customized Web offers or telemarketing scripts.

In addition, an intelligent extranet might allow customers or suppliers to access their account information online and perform various types of transactional applications. The intelligent extranet might also provide an aggregated view of customer or supplier data so that customers and suppliers can benchmark their performance against the overall group.

To support an operational view of customer and supplier information, companies are deploying operational data stores (ODS) or operational caches to update and maintain real-time customer profiles. These caches are either centralized or synchronized so that all touchpoint applications access the same real-time data.

Challenge: The data warehousing manager needs to differentiate the roles that a staging area, data warehouse, data marts, operational data store, intelligent extranets and external service providers play in delivering information within an intelligent business. In most cases, all these data structures and applications need to work in concert to deliver the information to the users and applications that need it when they need it.

Rule 4: Federate
A mantra for the intelligent business in the e-world is to provide the right information to the right user or app in an acceptable time frame. The problem here is that the information intelligent e-business users and applications need is rarely maintained or stored in one place in the same format or structure.

For example, the operational cache mentioned above may pull information from data warehouses, data marts, internal operational systems and external data. The cache may collect and process all this information in batch overnight or dynamically access that data in response to a specific query. For instance, when a new customer registers at your Web site, you can append their profile with demographic data, credit information or other relevant data downloaded dynamically from third-party information maintained by third-party data aggregators, such as Acxiom or Experian.

In addition, external customers may want to look at their balances and transactions for multiple accounts running in different systems, plus trending information over time for their portfolio as a whole. A bank loan officer may want to review an online customer's payment schedule and transactions, as well as various documents such as contracts, property descriptions and agency ratings, all of which are maintained in different systems.

Challenge: Data warehousing managers need to help their companies identify the most appropriate data structures in which to store information, given its intended use by the intelligent e-business. Then they must provide information pathways for gathering and integrating that information, usually on the fly. For users, this means creating a robust semantic layer that shields users from the complexity and multiplicity of back-end systems. Ideally, users never need to know that their query requires retrieving and integrating data from, for example, a data warehouse, a data mart, an operational system and an external provider.

Rule 5: Consolidate
It is difficult to create a comprehensive customer profile when there is no consistent way to maintain and reconcile customer records. The same problem also afflicts product data and descriptions. Today, most companies maintain customer and product information in haphazard ways.

Typically, each touchpoint app collects customer data in different formats with different levels of validation and scrubbing. Moreover, many customers provide inconsistent or erroneous data about themselves, sometimes intentionally, or their information changes because they have moved, changed jobs or changed marital status. The Web exacerbates these problems because of the anonymous nature of most Web traffic. In addition, the regularity of mergers and acquisitions makes maintaining accurate customer and product profiles a Herculean task.

An intelligent business addresses these problems by developing a strategy for creating unique customer identifiers across functional departments and lines of business, and then validating and reconciling customer data to create accurate profiles.

Challenge: Because of their experience in integrating diverse sets of data, the data warehousing team is the obvious choice to help the intelligent e-business establish the processes and structures for developing and maintaining unique customer records. Ideally, firms need to determine a common definition and format for capturing customer information, and implement these changes in all touchpoint and operational apps. More realistically, data warehousing professionals need to apply industrial-strength householding tools to clean, match and consolidate customer records using a unique identifier that works across all applications.

Rule 6: Associate
E-business users do not have time to conduct multiple searches to find the information they need to perform their jobs. This is especially true for customers or suppliers, who will take their business elsewhere if they cannot get all the information they need to make a product choice quickly, conduct routine business or troubleshoot problems.

Consequently, intelligent businesses in the e-world must be able to associate information from diverse sources, including numeric, text, XML, image, audio, spreadsheets and other data. An intelligent search engine needs to query a corporate-wide meta data repository that maintains indexed references to information objects stored in multiple systems. Ideally, this engine presents users with a coherent report of related content rather than a long list of result items. The engine also does not display items to users who are not authorized to view them.

Challenge: Data warehousing managers need to help build or deploy this universal query mechanism and meta data repository. More than likely, data warehousing managers will be required to create an XML interface to the data warehouse's meta data repository, or underlying data, as well as content management systems. This will enable search engines embedded within enterprise information portals to query across SQL and non-SQL data sources to deliver results within an XML-compliant interface.

Rule 7: Automate
An intelligent business establishes an information life cycle for managing customer interactions. This cycle captures how individual customers react to sales, marketing and service campaigns, and feeds that information back into their profiles for further analysis and action. This feedback loop lets intelligent e-businesses increase their knowledge of customer behavior and market dynamics to continuously refine campaigns and personalization strategies.

More importantly, intelligent businesses seek to automate this cycle to deliver real-time cross-sell or up-sell offers, or highly tailored information. The goal is to capture customer behavior at the point of interaction, feed that data into a customer's profile, and then apply the most appropriate business rules given the context of the interaction. The result is a dynamically generated and highly tailored recommendation delivered via a Web page, call-center representative or retail salesperson.

Interestingly, near real-time personalization via E-mail, pagers or cellphones may be more powerful than real-time personalization. For example, a national pizza chain might send a custom message to your pager to see if you want to reorder the pepperoni and onion pizza you purchase every Wednesday night after the kids' soccer game. You simply click "OK," and the order is routed to the nearest affiliated pizza outlet, which cooks and delivers your pizza at the predetermined time.

Challenge: The most compelling forms of personalization require knowledge about customer attributes and past interactions that can be analyzed using sophisticated statistical tools. These tools are then used to create models of customer behavior. The rules can then be applied in real time to new or existing customers who exhibit new types of behavior.

Closed-loop applications for managing intelligent customer interactions require the integration of many different types of tools, data structures and applications. Few firms have the resources or expertise to build such applications. Fortunately, e-commerce and business intelligence vendors are teaming up to provide packaged apps that marry customer intelligence capabilities with real-time, closed-loop customer interaction software.

Rule 8: Proliferate
The intelligent business in the e-world believes information is the currency of the new global marketplace and that the Web is the best channel for giving users, both inside and outside the corporation, access to the information and apps they need to perform their jobs.

The Web makes it affordable for firms to democratize information access. Multitier, Web-based analytical tools support access control features to maintain adequate security while opening up the data warehouse and other sources to external users.

Challenge: The goal for data warehousing managers is to plan for exponential growth in the number of users. Managers should examine the capabilities and architectures of Web query, reporting and analysis products to ensure that they support distributed processing, load balancing and failover features, and minimize network traffic between Web clients and servers.

Rule 9: Facilitate
To facilitate end-user access in an e-business environment, companies will need to make the process of querying or "searching" as intuitive as possible. Ideally, the process should mimic the Web paradigm for searching for information on the Internet. Also, it follows that query results should come back within a few seconds, not a few minutes or hours. This will be as true for complex queries of multiple sources as it is for simple queries.

Challenge: Data warehousing managers will need to press analytical tools vendors to deliver simplified user interfaces, most of which are designed for power users. These tools also need a global semantic layer (mentioned above) that shields users from the complexity of querying multiple data sources and multiple data types. To facilitate fast query response times, the tools must support robust, multitier architectures and caching mechanisms.

Rule 10: Placate
Finally, e-businesses must respect the privacy of their customers and suppliers. Companies should monitor federal regulatory activity governing Internet privacy and stay a step ahead of impending regulations. More importantly, they should monitor their customers' opinions about privacy issues and err on the conservative side when establishing new programs for collecting and using customer information.

Ideally, companies should establish and adhere to a privacy policy that defines why they collect customer information, how they manage it and what they do with it. Companies should also favor an "opt-in" policy that restricts companies from sending out marketing information unless customers specifically request such material, rather than an "opt out" policy that gives companies greater license to contact users.

Challenge: Data warehousing managers need to help companies preserve the integrity of customer information by applying adequate levels of security and rules-driven processes to prevent unauthorized access and use by individuals or applications. They should also provide users with a way to browse their personal profiles online and to correct errors.

E-business has already transformed the business and computing landscape. However, the market is only in the first stages of the total e-business revolution. The next stage involves marrying e-commerce with business intelligence to create the intelligent e-business. When this happens, companies will be able to learn from each customer interaction and use this knowledge to continuously refine sales, marketing and service campaigns, and to streamline supply chains.

Data warehousing professionals need to play a key role in transforming their companies from information archivists into intelligent e-businesses. Managers who understand and adhere to the 10 rules for building an intelligent business in the e-world can become agents of change within their organizations and valued members of the executive team.