In-Depth

Turning rules into requirements

• By Ellen Gottesdiener
• June 5, 2001

The term "business rule" is used to discuss both the requirements analysis phase and the logical or physical design phase of software development. When people talk about requirements analysis, the conversation usually centers around what the business rules are, how to elicit them, and how to extend them forward to the design and implementation phases of a project. When it comes to the logical or physical stage of software development, the application development manager's chief concern is where to place business rules so that they remain very flexible but do not degrade application performance.

Analysts and developers use a process called
business-rule modeling to elicit, analyze, validate and document business rules. These tasks enable implementation of the right business, and exemplify the prime directive of requirements analysis: To define the right requirements. Further, business-rule modeling is a prerequisite for implementation, regardless of the specific technology used.

Business rules -- broadly defined as the policies and constraints of the business -- provide the knowledge behind any business structure or process. There are also numerous, non-standardized but valid taxonomies
for business rules [see "Business rules show power, promise," by Ellen Gottesdiener, ADT, March 1997, p. 36]. Regardless of the taxonomy used, however, business rules are fundamental and must be known to yield software that meets the needs of business partners.

A "term" provides the basis for business-rule modeling. The definition of a term used in a business (such as 'customer,' 'claim,' 'registration' or 'case') provides the foundation for all other business rules -- regardless of the taxonomy chosen. "Definitions can be considered constraints, since they place boundaries on a concept," explained James Odell, an independent consultant in Ann Arbor, Mich. Terms are connected together (for example, "facts" such as customer 'orders' product or claim 'contains' incident-date) and rules are based on these fundamentals.

Beyond these fundamentals are the complex business rules that are the most elusive and challenging to capture. These rules can comprise many sub-rules or clauses that are rules themselves. Thus, a rule such as: "a customer order, which is packed and for which the customer has at least one unpaid but filled order, cannot be shipped" is replete with sub-rules. In fact, rules contain rules, making them a "composite."

Motivation for a business-rules approach

Driving the desire to use a business-rules approach is the need for flexibility, speed and, in some cases, the deeper recognition that business rules are an asset because they are the core knowledge of the business.

"What's unique about the business-rules approach is the realization that business rules are an independent variable of data, process, people and location," said John Zachman, author of the Zachman framework, an enterprise architecture framework, and chairman of the Zachman Institute for Framework Advancement (ZIFA). "We are realizing that unless you deliberately get the rules out, they get embedded in other things, which inhibits flexibility. The key to flexibility is to de-couple independent variables. Unless you figure this out, 'flexibility' is simply a platitude."

Added Zachman, "You cannot change the rule unless it is identified independently. With a business-rules approach, we are recognizing that having two or more independent variables hard-bound together doesn't promote the ability to change."

This need for change is further coupled with a desire to accelerate business processes, create new products and services, and change them quickly and correctly. "An important driver for a rule- [or knowledge-] based approach is the emerging need to automate back-office processes in banking and insurance. An application [for a loan or insurance] has to be assessed directly. There is no time to wait for interruption by people such as underwriters anymore," said Leo Hermans, Ph.D., senior consultant on knowledge- and rules-based application projects at Everest BV, The Netherlands.

"Another driver is the need to add intelligence to front-office-advising applications supporting sales, agents or even customers via the Internet. Still another is the need to make policies and regulations, that change frequently, explicit and therefore maintainable by experienced users," he said.

An example of this can be found at American International Group (AIG), New York City. "The trend toward the Internet, as well as our product strategy, has moved AIG into a more flexible application architecture, which includes having business rules in a separate component," said May Abraham, director, CTO-Knowledge Management Group at AIG. "You cannot add new products quickly if they are encoded, so we need rules in separate components."

The concept that business rules are, like data, an enterprise asset is becoming a more accepted notion. "An organization should be aware that knowledge is an asset that is at least as critical as information. This means that knowledge management must be made explicit in the organization, for example, by having knowledge managers," said Hermans at Everest BV. "Rules-based systems are only one of the dif-
ferent kinds of solutions for knowledge bottlenecks." Other tools such as groupware, performance support and human resource metrics are also useful, he noted.

According to Gladys Lam, principal at Business Rule Solutions in Houston, "The knowledge of the business is embedded in the tactics used to drive the business; in the policies enforced by the business; the risk tolerance of the business; and the rules implemented by the business."

But others believe that business rules do not need separate attention -- if you are doing the analysis correctly. "In OO [object-oriented] analysis, you capture data and behavior together. In this context, business rules are part of the behavior of objects, so there is nothing special or different about them," said Martin Fowler, an independent consultant in Melrose, Mass. "This is part of what we've always done in OO analysis. It's not a separate thing."

Business-rules approaches

The manner in which business rules are captured -- as independent artifacts during requirements analysis, as enterprise knowledge using a top-down approach, or as elements embedded in requirement analysis models -- varies depending on project motivations, the characteristics of the business rules themselves, and business goals and objectives.

A top-down approach is best done using the Zachman Framework as a guide. "Business rules derive from Column 6 [of the framework -- the "why" or "motivation" column -- ], which is an expression of your motivation or value system," noted Zachman. Business Rule Solutions' Lam and Ron Ross, also a principal at Business Rule Solutions, advocate a Policy Charter to document the motivation for tactics and business policies. Such an approach "puts technology more in line with the business and captures business knowledge," said Lam.

But not all projects are able to start at the top. Longs Drugs, Walnut Creek, Calif., implemented a business-rules-based application to manage drug dispensing at stores across the U.S. The company uses location-specific rules loaded into tables that are then "pushed" to locations. "Business rules are treated as parameters whose definition resides as a value in a database," said David Plotkin, Longs' senior data administrator.

The idea of explicitly capturing business rules was introduced by Longs' IT group and quickly embraced by business users. A detailed business-rule meta model was created to store aspects of the business rules (business objects that enforce the business rule, locations where the business rule needs to be enforced, and so on). Business rules were not mapped to business objectives or tactics per se, but to all application models to ensure that the software would be built correctly.

An approach to business-rule modeling advocated by Ellen Gottesdiener of EBG Consulting Inc., Carmel, Ind. -- and evolved from a collaboration with Barbara von Halle of Mendham, N.J.-based Knowledge Partners Inc. -- is oriented to business-rule requirements engineering. This approach recommends that a modeling or business goal route for eliciting business rules be selected depending on project objectives and how central business rules are to the application. Business rules are then mapped against business objectives if business reengineering is explicitly desired. The approach emphasizes the critical need for active business sponsorship to support eliciting, validating and perhaps reengineering business rules.

Mapping business rules to business objectives and tactics is a form of validation, which ensures that requirements can be traced back to the original need ("Am I building the right software?"). Verification, on the other hand, ensures that software correctly implements the functionality and underlying business rules ("Am I building the product right?").

Validating that business rules map to objectives and goals is highly recommended by Plotkin at Longs Drugs. "To do it right, you need to understand why you have this rule. In our case, it was due to a regulation, corporate policy or to compensate for inadequacies in our old system," he said. "One thing we did not do because of project timing pressure was to go back and map to business objectives. In retrospect, I would recommend it."

According to Kirk Wilson, there are two camps in the modeling community. "One is the 'business-rules' community, many coming from the data and database world, with a strong meta model for capturing business rules," said Wilson, manager of advanced technology in the Redwood City, Calif.-based Knowledge Systems Lab at Platinum Technology Inc. "Another is the object-oriented community, which has virtually no meta model to address rules as such, but which addresses business rules only within the context of an object model. These two communities have got to come together."

He points out that the best approach might be to combine the strengths of these differing perspectives. "The knowledge engineering community has been using rules for problem solving and performing a business process," said Wilson. "The business-rules community tends to look at business rules as constraints to control processes."

Rule-oriented practitioners with experience and background in artificial intelligence and knowledge engineering draw on the CommonKADS (Knowledge Acquisition and Design Structure) methodology that emerged in Europe in the early 1980s. CommonKADS makes use of patterns and models to describe knowledge-intensive tasks. Wilson, Everest BV's Hermans and others with this background make a useful distinction between rule-based and rule-constrained processes. Rule-based systems are very complex and tend to solve a problem. They originate in the application of a policy or a regulation (rating an insurance policy or determining an offender's incarceration time, for example), which exists to enforce rules. On the other hand, rule-constrained processes already exist as part of some business process (class registration, submitting expenses) and need to have certain constraints enforced. These rules are often tightly coupled to the data and business transactions. The approach used to subsequently model business rules may vary depending upon on the distinctions between the two processes.

"When we are just talking about constraint-type rules, such as those on structural and behavioral models, then these can be modeled easily in data or object models and in concepts such as pre- and post-conditions," said Hermans at Everest BV. "In rule-based systems, the knowledge [reasoning strategy, as well as rule sets underlying reasoning steps] has to be made explicit as a first-class citizen in modeling. Only by making knowledge explicit in this way, and thus giving processing context to business rules, can knowledge be
represented in a way that enables easy maintenance when policies or regulations change," he said.

Whether you are taking a top-down, project-specific, rule-based or rule-constrained approach, the notion of explicitly capturing, validating and verifying business rules can be highly advantageous. "The business-rules approach is a different and useful enough paradigm that it needs to be added to our kit of tools," noted consultant Odell.

Requirements analysis and business rules

Software projects follow patterns to accomplish
the software development process. Although the specifics of this process pattern vary (waterfall, spiral, staged delivery and so on), a constant is the sets of activities that must be done. These activities are grouped into stages or phases that can be categorized as follows: planning, requirements analysis, design, construction, testing and implementation.

The aim of the requirements analysis phase is to understand business partners' needs. These may range from solving a problem to achieving a goal for which technology may provide a solution. Needs and goals, defined at a business level, are translated into business requirements that may include new or altered business processes. Requirements to be solved by software are then allocated to the IT department. As the requirements analysis phase concludes, requirements allocated to IT are further allocated to either software or hardware. In this way, requirements analysis deals with the 'problem space,' while design deals with the 'solution space.' Because business users are the source of deliverables, when they work in collaboration with IT and invest their time in requirements discovery and analysis activities, this phase yields higher quality requirements.

A business requirement -- such as "provide capability to order investigation reports for all new claims" -- comprises functional and non-functional requirements. Functional requirements are the capabilities and behaviors that must be performed ("an investigation shall be ordered within 24 hours of notification of a violation," for example), while non-functional requirements have to do with the constraints around those behaviors ("investigation reports must be accessible to all corrections counselors via the intranet").

Business rules are at the heart of functional requirements. They are what the functional requirement knows -- the decisions, guidelines and controls that are behind the functionality. Given the example above ("an investigation shall be ordered within 24 hours of notification of a violation"), rules include the triggering event (a phone call,
E-mail or fax) and constraints ("if offender has a
repeat offense claim, notify parole board").

Engineering business rules into requirements

As imperfect as we humans are, we know that when we start to analyze requirements they will be unclear, inconsistent, incomplete and perhaps irrelevant. Therefore, at the conclusion of the requirements analysis phase, requirements must be clarified, baselined, signed off and documented in a Software Requirements Specification.

USAA, a private auto and homeowner insurance writer in San Antonio, began explicitly capturing business rules as part of its business requirements. Having recognized the need to have reproducible software development processes, USAA began by defining and tracing requirements that would yield a minimum amount of ambiguity for both IT and business customers.

As part of this effort, said IT Systems Analyst Dave Butler, a requirements management process has been established that includes tools such as QSS's DOORS
requirements management tool. USAA's requirements tracing follows a path that includes: business need (a general statement describing a business objective or business problem); business requirement (a formal statement of a business need to solve a problem, or to achieve a goal/objective); and business rules (detailed description of events, behaviors, actions and constraints that comprise a business requirement).

"A project with one thousand high-level business requirements can yield four thousand or so rules," said Butler, who advocates use of an automated requirements management tool to help manage this process and volume. For example, a business rule such as "All preliminary background checks shall be performed before opening a claim" or "Determine whether potential protection against a financial loss is covered under the terms of an insurance policy for a particular loss event" could be spawned from business requirements, and then need to be captured, validated and tested before implementation.

At USAA, facilitated workshops are used to capture business needs, requirements and rules. "Our business customers participate in the requirements, development and design [phases] to assure that testing of systems addresses business needs," said Butler.

Because "business people prefer text-based requirements and system people prefer use cases," USAA uses both approaches. The company also builds acceptance criteria for testing business requirements rules. "We can take a business rule or requirement and trace it down to the lowest test case," said Butler. "We want to verify the rule and, if the rule changes, see what test cases are impacted and how to change them."

Models and business rules

Models permit us to communicate, visualize and abstract things. Visual and text models help immensely with requirements analysis, and are the reason behind the slew of modeling tools and methodologies available today.

In general, approaches to modeling functional requirements include: process-oriented (functional decomposition, workflows, DFDs, process maps and so on), data-oriented (data model), object-oriented (objects with encapsulated state and behavior) and control-oriented (events and statecharts). But regardless of the symbols used, each of these modeling approach methods depends upon business rules.

No matter what modeling notation is used, business rules still need to be elicited and verified. The Unified Modeling Language (UML), for example, provides two choices for explicitly modeling business rules. The first is via an extension called a 'constraint,' which is a "semantic condition or restriction." This allows the modeler to more precisely define constraints on structural (class/object) models beyond those already expressed in the semantics of the model (association, multiplicity, generalization) or those available as pre-defined UML constraints (complete, disjoint or overlapping, and so on). A UML constraint takes the form of freeform text in braces (or optionally displayed inside a 'note' symbol) attached to any UML modeling element such as a class.

The second option is the Object Constraint Language (OCL), which is part of the UML. A constraint is defined here as a "declaration of what must be true.⁶" The OCL reads like a combination of object-oriented languages with predicate logic thrown in. It is a specification level that
requires a well-defined class model and knowledge of the language itself. It is not useful for business rules that need to be captured during requirements analysis in collaboration with business customers.

"The OCL is a rule language, but we want something that the user can understand," said consultant Odell. "If you can read first-order predicate logic, then fine. In fact, you could use Prolog. But business rules must be readable to business people." The current OCL standard formalizes only the structural model because, according to Odell, it is harder to formalize the behavior part, which is currently in development.

A useful distinction can be made between a business rule specified at a requirements analysis level vs. one specified at a design or implementation level. Business rules specified during requirements analysis can be represented in a variety of ways:

• as natural language text;
• as natural language text templates;
• via symbols of existing modeling languages (the cardinality symbol of data or object modeling; the condition portion of the activity diagram; the constraint on any UML model; events, guards and actions on a statechart diagram, or entity life history diagrams);
• by extending the graphical model using natural or formal language (Odell's extensions);
• as decision tables and decision trees; and
• as a combination of the above.

"It's becoming clear that how business rules are expressed externally should be separated from how they are represented internally," said Ross at Business Rule Solutions. "Internal means transparent to the specifier, but visible to analysis tools for conflict analysis or for run-time processing by a rule engine or business logic server." For this level, business rules can be represented using:

• OCL,
• Predicate logic,
• the Ross Method⁷ and
• Halpin's Object Role Modeling (ORM).

Each requirements analysis model has a general set of semantics and may depict behavior, structure, dynamics or a combination thereof. At its core, the model expresses business rules, although some are "hidden" within the modeling semantics. Some models are more expressive of business rules than others. For example, the UML statechart diagram includes modeling elements for trigger, condition and action, while the UML activity diagram includes modeling elements for triggers, guards (conditions), decisions and actions. These provide richer opportunities to
express business rules, but are still not as expressive or direct as natural language is for business users.

Business rule elicitation

One of the chief challenges in any business-rule approach is to elicit the business rules. While business experts are the source of the rules, they sometimes do not agree on or know what the business rules are or should be. One best practice that emerges is the use of facilitated workshops (JAD-like structured workshops). This is done in the context of building other models that supplement business rules.

"One does not ask the business to 'do business rules,'" said consultant Odell. "Rather, business rules are elicited in the context of diagram building. They emerge when specifying events, triggers, processes, classes, relationships and so on."

Longs Drugs, for example, conducted intensive workshops over a six-week period with business customers to capture requirements and weave detailed modeling and prototyping into the process. The models created in the workshops included use cases, natural language business rules and a class model. Models were carefully extended and analyzed, with each attribute having a rigorous definition. Use case text models included business rules in natural language. The team also compared existing system and vendor applications to the business rules from the workshops to ensure the correctness of the business rules.

Tools further supplemented the process. Use cases were written in Word, the class model was kept in Select Enterprise and the business-rule meta model was stored in ERwin. An evolutionary prototyping process was used to validate the business rules, and accompanying workflows were modeled by creating PowerBuilder-generated screens loaded with sample data.

"You need to have an environment that supports modeling," said Plotkin at Longs Drugs. "The resources, tools and money to do all this, along with the end user's inclination to do it" are critical for success.

Elicitation may require a variety of techniques, however. Coppell, Texas-based Mannatech -- a nutraceutical (products that are a cross between dietary supplements and pharmaceuticals) firm -- uses three techniques to capture business rules and other requirements, said Bill Councill, a systems and process engineering manager in the IT department. Among these are: 1) having trained people from the Compliance department write requirements; 2) capturing requirements in facilitated workshops along with use cases; and 3) having business analysts interview business experts. In each case, the business rules and other requirements are managed in the requirements management tool and follow the rigor outlined by the IEEE standard.

Techniques that require less business user interaction can also be appropriate. "To capture and elicit business rules, we use several kinds of interviews with expert users, study available documentation, and build prototypes and task-/
domain-specific knowledge editors with which expert users can enter rules themselves," said Hermans at Everest BV.

Such knowledge acquisition techniques have their roots in artificial intelligence. These techniques include use case analysis, process description, interviews and observation, said Nancy Wood, director of implementation services in Platinum Technology's Rules-Based Consulting Practice.

Wood, who works with many insurance companies, first creates a structural model using business objects such as driver, vehicle and claim. "Then we generate 'pseudo' rules with scenarios and use cases, and write the business rules in terms of business objects." Platinum's Aion product is then used to prototype during knowledge acquisition. "This helps a lot when a business user is having trouble explaining or representing their knowledge," she said. "We also get policy guidelines and manuals and find that users often don't use them in the day-to-day running of the business."

Business-rule traceability

Erroneous, incomplete, ambiguous and conflicting requirements are the source of the majority of defects in software development. Traceability from requirements to design, testing and implementation provides numerous benefits to the business customer and IT, including: a mechanism to detect conflicts; a repository to understand the impact of change; formalization and justification of product features and constraints; reduced risk of requirements defects; maintenance assistance through faster location of where business rules are physically implemented; and a repository for capturing and understanding requirements attributes.

Consider a "defect" in a functional requirement. What might it be? The answer is often an error in the business rules (for example, it is incomplete, incorrect, unclear and/or inconsistent). How-ever, when business rules are added as a distinct requirement type, with attributes of their own, risk is reduced. Further, when treated in this manner, requirements verification is more likely to be conducted. Yet this also means that there must a way to test requirements.

One way to accomplish traceability is to use a business-rules meta model. At Longs Drugs, for example, Plotkin and his colleagues developed a business-rule meta model by mapping each business rule elicited in the user workshops to properties on the class model. During the requirements analysis phase, an Excel spreadsheet was used to keep track of the details (another alternative would be to use a requirements management tool). The actual business-rule meta model stores the real values and links to business objects in production.

Neville Haggerty, president of Ports- mouth, N.H.-based Compedia, a data quality assessment and business requirements firm, worked with this author on a business-rule project. As the project's chief business-rule capture tool architect, Haggerty extended Popkin Software's System Architect modeling tool repository to support business-rule capture in facilitated workshops. "With System Architect, it was easy to change the repository meta model to add a business-rule object, specify its properties and add relationships to other objects like data elements," said Haggerty.

"In addition, once the business rules are in the tool, the client can go the next step and create models for class, data and process flow in System Architect," he added. "With the business models stored in the same repository as the business rules, analysts can relate business rules to the objects that implement the rules in the models. This provides business-rule traceability."

At Logica, a Lexington, Mass.-based systems integration and consulting firm, Requirements Analyst Mary Walker is capturing requirements and associated models using Rational's RequisitePro and the Rose modeling tool. Using the "two column" use case format (actor behavior and system response), Walker is documenting formal requirements derived from use cases, as well as atomic, data-centric business rules in a separate Word document. The requirements in RequisitePro, she said, reference the business-rule document. "This helps us to focus on traceability from requirements through integration," said Walker.

The ideas behind change management processes and tools are the same as those needed for business rules. "Rule management is critical. We need to be able to distribute, store and synchronize an enterprise rule system across many people in many countries," said Abraham at AIG.

Regulated businesses, or ones in which the business rules are very complex, should consider using a requirements management tool based on a project-specific business-rule meta model. The meta model ensures that the business rules trace to the important things, for example business objects, models, owner, documents and so on. The requirements management tool should be extensible in order to handle these attributes, as well as business rules, as a distinct requirement type.

Another approach may also be appropriate -- obtaining traceability via testing. According to consultant Fowler, he prefers this simpler approach to business rule and functional requirements. "I don't go for requirements traceability -- [it's] a lot of bureaucracy," he noted. Using tests to handle traceability allows each requirement and business rule to be the source of test cases.

Tread with caution

Business-rules capture seems an obvious, and sensible, activity to include in IT projects. But pitfalls exist. Explicitly capturing business rules often reveals organizational "dirt": rules that are inconsistent, in conflict, inefficient, redundant, non-standardized, non-compliant with regulations or company policy, or that have no owner. "Cultural problems are a bigger problem than capturing, engineering and measuring business rules," said Zachman, "because it means getting people to change their behavior."

Thus, business-rules capture should be undertaken with a bit of forewarning. But opportunity accompanies the danger. Not only does business-rule requirements engineering help us build software right, it can help us assist our business partners in building the right software. When led by senior business sponsorship, exposing business rule "dirt" can help us rethink those rules to reveal and share knowledge and to optimize the business.