In-Depth
Turning rules into requirements
- By Ellen Gottesdiener
- June 5, 2001
The term "business rule" is used to discuss both the
requirements analysis phase and the logical or physical design phase of software development. When people talk
about requirements analysis, the conversation usually centers around what the business rules are, how to elicit
them, and how to extend them forward to the design and implementation phases of a project. When it comes to the
logical or physical stage of software development, the application development manager's chief concern is where
to place business rules so that they remain very flexible but do not degrade application performance.
Analysts and developers use a process called
business-rule modeling to elicit, analyze, validate and document business rules. These tasks enable implementation
of the right business, and exemplify the prime directive of requirements analysis: To define the right requirements.
Further, business-rule modeling is a prerequisite for implementation, regardless of the specific technology used.
Business rules -- broadly defined as the policies and constraints of the business -- provide the knowledge behind
any business structure or process. There are also numerous, non-standardized but valid taxonomies
for business rules [see "Business rules show power, promise," by Ellen Gottesdiener, ADT, March 1997,
p. 36]. Regardless of the taxonomy used, however, business rules are fundamental and must be known to yield software
that meets the needs of business partners.
A "term" provides the basis for business-rule modeling. The definition of a term used in a business
(such as 'customer,' 'claim,' 'registration' or 'case') provides the foundation for all other business rules --
regardless of the taxonomy chosen. "Definitions can be considered constraints, since they place boundaries
on a concept," explained James Odell, an independent consultant in Ann Arbor, Mich. Terms are connected together
(for example, "facts" such as customer 'orders' product or claim 'contains' incident-date) and rules
are based on these fundamentals.
Beyond these fundamentals are the complex business rules that are the most elusive and challenging to capture.
These rules can comprise many sub-rules or clauses that are rules themselves. Thus, a rule such as: "a customer
order, which is packed and for which the customer has at least one unpaid but filled order, cannot be shipped"
is replete with sub-rules. In fact, rules contain rules, making them a "composite."
Motivation for a business-rules approach
Driving the desire to use a business-rules approach is the need for flexibility, speed and, in some cases, the
deeper recognition that business rules are an asset because they are the core knowledge of the business.
"What's unique about the business-rules approach is the realization that business rules are an independent
variable of data, process, people and location," said John Zachman, author of the Zachman framework, an enterprise
architecture framework, and chairman of the Zachman Institute for Framework Advancement (ZIFA). "We are realizing
that unless you deliberately get the rules out, they get embedded in other things, which inhibits flexibility.
The key to flexibility is to de-couple independent variables. Unless you figure this out, 'flexibility' is simply
a platitude."
Added Zachman, "You cannot change the rule unless it is identified independently. With a business-rules
approach, we are recognizing that having two or more independent variables hard-bound together doesn't promote
the ability to change."
This need for change is further coupled with a desire to accelerate business processes, create new products
and services, and change them quickly and correctly. "An important driver for a rule- [or knowledge-] based
approach is the emerging need to automate back-office processes in banking and insurance. An application [for a
loan or insurance] has to be assessed directly. There is no time to wait for interruption by people such as underwriters
anymore," said Leo Hermans, Ph.D., senior consultant on knowledge- and rules-based application projects at
Everest BV, The Netherlands.
"Another driver is the need to add intelligence to front-office-advising applications supporting sales,
agents or even customers via the Internet. Still another is the need to make policies and regulations, that change
frequently, explicit and therefore maintainable by experienced users," he said.
An example of this can be found at American International Group (AIG), New York City. "The trend toward
the Internet, as well as our product strategy, has moved AIG into a more flexible application architecture, which
includes having business rules in a separate component," said May Abraham, director, CTO-Knowledge Management
Group at AIG. "You cannot add new products quickly if they are encoded, so we need rules in separate components."
The concept that business rules are, like data, an enterprise asset is becoming a more accepted notion. "An
organization should be aware that knowledge is an asset that is at least as critical as information. This means
that knowledge management must be made explicit in the organization, for example, by having knowledge managers,"
said Hermans at Everest BV. "Rules-based systems are only one of the dif-
ferent kinds of solutions for knowledge bottlenecks." Other tools such as groupware, performance support and
human resource metrics are also useful, he noted.
According to Gladys Lam, principal at Business Rule Solutions in Houston, "The knowledge of the business
is embedded in the tactics used to drive the business; in the policies enforced by the business; the risk tolerance
of the business; and the rules implemented by the business."
But others believe that business rules do not need separate attention -- if you are doing the analysis correctly.
"In OO [object-oriented] analysis, you capture data and behavior together. In this context, business rules
are part of the behavior of objects, so there is nothing special or different about them," said Martin Fowler,
an independent consultant in Melrose, Mass. "This is part of what we've always done in OO analysis. It's not
a separate thing."
Business-rules approaches
The manner in which business rules are captured -- as independent artifacts during requirements analysis, as
enterprise knowledge using a top-down approach, or as elements embedded in requirement analysis models -- varies
depending on project motivations, the characteristics of the business rules themselves, and business goals and
objectives.
A top-down approach is best done using the Zachman Framework as a guide. "Business rules derive from Column
6 [of the framework -- the "why" or "motivation" column -- ], which is an expression of your
motivation or value system," noted Zachman. Business Rule Solutions' Lam and Ron Ross, also a principal at
Business Rule Solutions, advocate a Policy Charter to document the motivation for tactics and business policies.
Such an approach "puts technology more in line with the business and captures business knowledge," said
Lam.
But not all projects are able to start at the top. Longs Drugs, Walnut Creek, Calif., implemented a business-rules-based
application to manage drug dispensing at stores across the U.S. The company uses location-specific rules loaded
into tables that are then "pushed" to locations. "Business rules are treated as parameters whose
definition resides as a value in a database," said David Plotkin, Longs' senior data administrator.
The idea of explicitly capturing business rules was introduced by Longs' IT group and quickly embraced by business
users. A detailed business-rule meta model was created to store aspects of the business rules (business objects
that enforce the business rule, locations where the business rule needs to be enforced, and so on). Business rules
were not mapped to business objectives or tactics per se, but to all application models to ensure that the software
would be built correctly.
An approach to business-rule modeling advocated by Ellen Gottesdiener of EBG Consulting Inc., Carmel, Ind. --
and evolved from a collaboration with Barbara von Halle of Mendham, N.J.-based Knowledge Partners Inc. -- is oriented
to business-rule requirements engineering. This approach recommends that a modeling or business goal route for
eliciting business rules be selected depending on project objectives and how central business rules are to the
application. Business rules are then mapped against business objectives if business reengineering is explicitly
desired. The approach emphasizes the critical need for active business sponsorship to support eliciting, validating
and perhaps reengineering business rules.
Mapping business rules to business objectives and tactics is a form of validation, which ensures that requirements
can be traced back to the original need ("Am I building the right software?"). Verification, on the other
hand, ensures that software correctly implements the functionality and underlying business rules ("Am I building
the product right?").
Validating that business rules map to objectives and goals is highly recommended by Plotkin at Longs Drugs.
"To do it right, you need to understand why you have this rule. In our case, it was due to a regulation, corporate
policy or to compensate for inadequacies in our old system," he said. "One thing we did not do because
of project timing pressure was to go back and map to business objectives. In retrospect, I would recommend it."
According to Kirk Wilson, there are two camps in the modeling community. "One is the 'business-rules' community,
many coming from the data and database world, with a strong meta model for capturing business rules," said
Wilson, manager of advanced technology in the Redwood City, Calif.-based Knowledge Systems Lab at Platinum Technology
Inc. "Another is the object-oriented community, which has virtually no meta model to address rules as such,
but which addresses business rules only within the context of an object model. These two communities have got to
come together."
He points out that the best approach might be to combine the strengths of these differing perspectives. "The
knowledge engineering community has been using rules for problem solving and performing a business process,"
said Wilson. "The business-rules community tends to look at business rules as constraints to control processes."
Rule-oriented practitioners with experience and background in artificial intelligence and knowledge engineering
draw on the CommonKADS (Knowledge Acquisition and Design Structure) methodology that emerged in Europe in the early
1980s. CommonKADS makes use of patterns and models to describe knowledge-intensive tasks. Wilson, Everest BV's
Hermans and others with this background make a useful distinction between rule-based and rule-constrained processes.
Rule-based systems are very complex and tend to solve a problem. They originate in the application of a policy
or a regulation (rating an insurance policy or determining an offender's incarceration time, for example), which
exists to enforce rules. On the other hand, rule-constrained processes already exist as part of some business process
(class registration, submitting expenses) and need to have certain constraints enforced. These rules are often
tightly coupled to the data and business transactions. The approach used to subsequently model business rules may
vary depending upon on the distinctions between the two processes.
"When we are just talking about constraint-type rules, such as those on structural and behavioral models,
then these can be modeled easily in data or object models and in concepts such as pre- and post-conditions,"
said Hermans at Everest BV. "In rule-based systems, the knowledge [reasoning strategy, as well as rule sets
underlying reasoning steps] has to be made explicit as a first-class citizen in modeling. Only by making knowledge
explicit in this way, and thus giving processing context to business rules, can knowledge be
represented in a way that enables easy maintenance when policies or regulations change," he said.
Whether you are taking a top-down, project-specific, rule-based or rule-constrained approach, the notion of
explicitly capturing, validating and verifying business rules can be highly advantageous. "The business-rules
approach is a different and useful enough paradigm that it needs to be added to our kit of tools," noted consultant
Odell.
Requirements analysis and business rules
Software projects follow patterns to accomplish
the software development process. Although the specifics of this process pattern vary (waterfall, spiral, staged
delivery and so on), a constant is the sets of activities that must be done. These activities are grouped into
stages or phases that can be categorized as follows: planning, requirements analysis, design, construction, testing
and implementation.
The aim of the requirements analysis phase is to understand business partners' needs. These may range from solving
a problem to achieving a goal for which technology may provide a solution. Needs and goals, defined at a business
level, are translated into business requirements that may include new or altered business processes. Requirements
to be solved by software are then allocated to the IT department. As the requirements analysis phase concludes,
requirements allocated to IT are further allocated to either software or hardware. In this way, requirements analysis
deals with the 'problem space,' while design deals with the 'solution space.' Because business users are the source
of deliverables, when they work in collaboration with IT and invest their time in requirements discovery and analysis
activities, this phase yields higher quality requirements.
A business requirement -- such as "provide capability to order investigation reports for all new claims"
-- comprises functional and non-functional requirements. Functional requirements are the capabilities and behaviors
that must be performed ("an investigation shall be ordered within 24 hours of notification of a violation,"
for example), while non-functional requirements have to do with the constraints around those behaviors ("investigation
reports must be accessible to all corrections counselors via the intranet").
Business rules are at the heart of functional requirements. They are what the functional requirement knows --
the decisions, guidelines and controls that are behind the functionality. Given the example above ("an investigation
shall be ordered within 24 hours of notification of a violation"), rules include the triggering event (a phone
call,
E-mail or fax) and constraints ("if offender has a
repeat offense claim, notify parole board").
Engineering business rules into requirements
As imperfect as we humans are, we know that when we start to analyze requirements they will be unclear, inconsistent,
incomplete and perhaps irrelevant. Therefore, at the conclusion of the requirements analysis phase, requirements
must be clarified, baselined, signed off and documented in a Software Requirements Specification.
USAA, a private auto and homeowner insurance writer in San Antonio, began explicitly capturing business rules
as part of its business requirements. Having recognized the need to have reproducible software development processes,
USAA began by defining and tracing requirements that would yield a minimum amount of ambiguity for both IT and
business customers.
As part of this effort, said IT Systems Analyst Dave Butler, a requirements management process has been established
that includes tools such as QSS's DOORS
requirements management tool. USAA's requirements tracing follows a path that includes: business need (a general
statement describing a business objective or business problem); business requirement (a formal statement of a business
need to solve a problem, or to achieve a goal/objective); and business rules (detailed description of events, behaviors,
actions and constraints that comprise a business requirement).
"A project with one thousand high-level business requirements can yield four thousand or so rules,"
said Butler, who advocates use of an automated requirements management tool to help manage this process and volume.
For example, a business rule such as "All preliminary background checks shall be performed before opening
a claim" or "Determine whether potential protection against a financial loss is covered under the terms
of an insurance policy for a particular loss event" could be spawned from business requirements, and then
need to be captured, validated and tested before implementation.
At USAA, facilitated workshops are used to capture business needs, requirements and rules. "Our business
customers participate in the requirements, development and design [phases] to assure that testing of systems addresses
business needs," said Butler.
Because "business people prefer text-based requirements and system people prefer use cases," USAA
uses both approaches. The company also builds acceptance criteria for testing business requirements rules. "We
can take a business rule or requirement and trace it down to the lowest test case," said Butler. "We
want to verify the rule and, if the rule changes, see what test cases are impacted and how to change them."
Models and business rules
Models permit us to communicate, visualize and abstract things. Visual and text models help immensely with requirements
analysis, and are the reason behind the slew of modeling tools and methodologies available today.
In general, approaches to modeling functional requirements include: process-oriented (functional decomposition,
workflows, DFDs, process maps and so on), data-oriented (data model), object-oriented (objects with encapsulated
state and behavior) and control-oriented (events and statecharts). But regardless of the symbols used, each of
these modeling approach methods depends upon business rules.
No matter what modeling notation is used, business rules still need to be elicited and verified. The Unified
Modeling Language (UML), for example, provides two choices for explicitly modeling business rules. The first is
via an extension called a 'constraint,' which is a "semantic condition or restriction." This allows the
modeler to more precisely define constraints on structural (class/object) models beyond those already expressed
in the semantics of the model (association, multiplicity, generalization) or those available as pre-defined UML
constraints (complete, disjoint or overlapping, and so on). A UML constraint takes the form of freeform text in
braces (or optionally displayed inside a 'note' symbol) attached to any UML modeling element such as a class.
The second option is the Object Constraint Language (OCL), which is part of the UML. A constraint is defined
here as a "declaration of what must be true.6" The OCL reads like a combination of object-oriented
languages with predicate logic thrown in. It is a specification level that
requires a well-defined class model and knowledge of the language itself. It is not useful for business rules that
need to be captured during requirements analysis in collaboration with business customers.
"The OCL is a rule language, but we want something that the user can understand," said consultant
Odell. "If you can read first-order predicate logic, then fine. In fact, you could use Prolog. But business
rules must be readable to business people." The current OCL standard formalizes only the structural model
because, according to Odell, it is harder to formalize the behavior part, which is currently in development.
A useful distinction can be made between a business rule specified at a requirements analysis level vs. one
specified at a design or implementation level. Business rules specified during requirements analysis can be represented
in a variety of ways:
- as natural language text;
- as natural language text templates;
- via symbols of existing modeling languages (the cardinality symbol of data or object modeling; the condition
portion of the activity diagram; the constraint on any UML model; events, guards and actions on a statechart diagram,
or entity life history diagrams);
- by extending the graphical model using natural or formal language (Odell's extensions);
- as decision tables and decision trees; and
- as a combination of the above.
"It's becoming clear that how business rules are expressed externally should be separated from how they
are represented internally," said Ross at Business Rule Solutions. "Internal means transparent to the
specifier, but visible to analysis tools for conflict analysis or for run-time processing by a rule engine or business
logic server." For this level, business rules can be represented using:
- OCL,
- Predicate logic,
- the Ross Method7 and
- Halpin's Object Role Modeling (ORM).
Each requirements analysis model has a general set of semantics and may depict behavior, structure, dynamics
or a combination thereof. At its core, the model expresses business rules, although some are "hidden"
within the modeling semantics. Some models are more expressive of business rules than others. For example, the
UML statechart diagram includes modeling elements for trigger, condition and action, while the UML activity diagram
includes modeling elements for triggers, guards (conditions), decisions and actions. These provide richer opportunities
to
express business rules, but are still not as expressive or direct as natural language is for business users.
Business rule elicitation
One of the chief challenges in any business-rule approach is to elicit the business rules. While business experts
are the source of the rules, they sometimes do not agree on or know what the business rules are or should be. One
best practice that emerges is the use of facilitated workshops (JAD-like structured workshops). This is done in
the context of building other models that supplement business rules.
"One does not ask the business to 'do business rules,'" said consultant Odell. "Rather, business
rules are elicited in the context of diagram building. They emerge when specifying events, triggers, processes,
classes, relationships and so on."
Longs Drugs, for example, conducted intensive workshops over a six-week period with business customers to capture
requirements and weave detailed modeling and prototyping into the process. The models created in the workshops
included use cases, natural language business rules and a class model. Models were carefully extended and analyzed,
with each attribute having a rigorous definition. Use case text models included business rules in natural language.
The team also compared existing system and vendor applications to the business rules from the workshops to ensure
the correctness of the business rules.
Tools further supplemented the process. Use cases were written in Word, the class model was kept in Select Enterprise
and the business-rule meta model was stored in ERwin. An evolutionary prototyping process was used to validate
the business rules, and accompanying workflows were modeled by creating PowerBuilder-generated screens loaded with
sample data.
"You need to have an environment that supports modeling," said Plotkin at Longs Drugs. "The resources,
tools and money to do all this, along with the end user's inclination to do it" are critical for success.
Elicitation may require a variety of techniques, however. Coppell, Texas-based Mannatech -- a nutraceutical
(products that are a cross between dietary supplements and pharmaceuticals) firm -- uses three techniques to capture
business rules and other requirements, said Bill Councill, a systems and process engineering manager in the IT
department. Among these are: 1) having trained people from the Compliance department write requirements; 2) capturing
requirements in facilitated workshops along with use cases; and 3) having business analysts interview business
experts. In each case, the business rules and other requirements are managed in the requirements management tool
and follow the rigor outlined by the IEEE standard.
Techniques that require less business user interaction can also be appropriate. "To capture and elicit
business rules, we use several kinds of interviews with expert users, study available documentation, and build
prototypes and task-/
domain-specific knowledge editors with which expert users can enter rules themselves," said Hermans at Everest
BV.
Such knowledge acquisition techniques have their roots in artificial intelligence. These techniques include
use case analysis, process description, interviews and observation, said Nancy Wood, director of implementation
services in Platinum Technology's Rules-Based Consulting Practice.
Wood, who works with many insurance companies, first creates a structural model using business objects such
as driver, vehicle and claim. "Then we generate 'pseudo' rules with scenarios and use cases, and write the
business rules in terms of business objects." Platinum's Aion product is then used to prototype during knowledge
acquisition. "This helps a lot when a business user is having trouble explaining or representing their knowledge,"
she said. "We also get policy guidelines and manuals and find that users often don't use them in the day-to-day
running of the business."
Business-rule traceability
Erroneous, incomplete, ambiguous and conflicting requirements are the source of the majority of defects in software
development. Traceability from requirements to design, testing and implementation provides numerous benefits to
the business customer and IT, including: a mechanism to detect conflicts; a repository to understand the impact
of change; formalization and justification of product features and constraints; reduced risk of requirements defects;
maintenance assistance through faster location of where business rules are physically implemented; and a repository
for capturing and understanding requirements attributes.
Consider a "defect" in a functional requirement. What might it be? The answer is often an error in
the business rules (for example, it is incomplete, incorrect, unclear and/or inconsistent). How-ever, when business
rules are added as a distinct requirement type, with attributes of their own, risk is reduced. Further, when treated
in this manner, requirements verification is more likely to be conducted. Yet this also means that there must a
way to test requirements.
One way to accomplish traceability is to use a business-rules meta model. At Longs Drugs, for example, Plotkin
and his colleagues developed a business-rule meta model by mapping each business rule elicited in the user workshops
to properties on the class model. During the requirements analysis phase, an Excel spreadsheet was used to keep
track of the details (another alternative would be to use a requirements management tool). The actual business-rule
meta model stores the real values and links to business objects in production.
Neville Haggerty, president of Ports- mouth, N.H.-based Compedia, a data quality assessment and business requirements
firm, worked with this author on a business-rule project. As the project's chief business-rule capture tool architect,
Haggerty extended Popkin Software's System Architect modeling tool repository to support business-rule capture
in facilitated workshops. "With System Architect, it was easy to change the repository meta model to add a
business-rule object, specify its properties and add relationships to other objects like data elements," said
Haggerty.
"In addition, once the business rules are in the tool, the client can go the next step and create models
for class, data and process flow in System Architect," he added. "With the business models stored in
the same repository as the business rules, analysts can relate business rules to the objects that implement the
rules in the models. This provides business-rule traceability."
At Logica, a Lexington, Mass.-based systems integration and consulting firm, Requirements Analyst Mary Walker
is capturing requirements and associated models using Rational's RequisitePro and the Rose modeling tool. Using
the "two column" use case format (actor behavior and system response), Walker is documenting formal requirements
derived from use cases, as well as atomic, data-centric business rules in a separate Word document. The requirements
in RequisitePro, she said, reference the business-rule document. "This helps us to focus on traceability from
requirements through integration," said Walker.
The ideas behind change management processes and tools are the same as those needed for business rules. "Rule
management is critical. We need to be able to distribute, store and synchronize an enterprise rule system across
many people in many countries," said Abraham at AIG.
Regulated businesses, or ones in which the business rules are very complex, should consider using a requirements
management tool based on a project-specific business-rule meta model. The meta model ensures that the business
rules trace to the important things, for example business objects, models, owner, documents and so on. The requirements
management tool should be extensible in order to handle these attributes, as well as business rules, as a distinct
requirement type.
Another approach may also be appropriate -- obtaining traceability via testing. According to consultant Fowler,
he prefers this simpler approach to business rule and functional requirements. "I don't go for requirements
traceability -- [it's] a lot of bureaucracy," he noted. Using tests to handle traceability allows each requirement
and business rule to be the source of test cases.
Tread with caution
Business-rules capture seems an obvious, and sensible, activity to include in IT projects. But pitfalls exist.
Explicitly capturing business rules often reveals organizational "dirt": rules that are inconsistent,
in conflict, inefficient, redundant, non-standardized, non-compliant with regulations or company policy, or that
have no owner. "Cultural problems are a bigger problem than capturing, engineering and measuring business
rules," said Zachman, "because it means getting people to change their behavior."
Thus, business-rules capture should be undertaken with a bit of forewarning. But opportunity accompanies the
danger. Not only does business-rule requirements engineering help us build software right, it can help us assist
our business partners in building the right software. When led by senior business sponsorship, exposing business
rule "dirt" can help us rethink those rules to reveal and share knowledge and to optimize the business.