Columns

e-Quality now!

• By Charles H. Trepper
• January 1, 2001

E-commerce applications are in desperate need of better quality control. Consider that, in many cases, a company's Web site is its first contact with new customers. If the Web site doesn't work, or the content is useless to a potential customer, a sale may be lost before the company is even aware of the potential new customer. Traditional quality assurance techniques focus on testing at or near the end of the development cycle. Demand for faster development times in the rapidly changing e-commerce space often means that Web sites are developed using iterative development techniques. In this case, testing at the end of the development cycle means the application has probably already been deployed, and customers may be seeing an untested or only partially tested application.

Web application quality has been sinking for quite some time. The advent of methodologies such as extreme programming and the exponential increase in the complexity of e-commerce apps are pushing quality control and assurance techniques to the limit. For example, I recently visited a content management software vendor's Web site. The vendor touts its software as providing "the highest quality content at Web speed." Yet the site has more than 40 broken links (that I took the time to test) and an abundance of data-retrieval errors. If this is development at Web speed, I'd hit the brakes.

Some customers, both internal and external, have simply given up on application quality. They accept poor software quality as a fact of life, and are unwilling to push IT organizations to do better. One needs only to examine the software packages consumers buy off the shelf or businesses use internally to see the awful state of software quality that people (both business-to-business customers and consumers) are willing to accept. It's human nature for businesses to deliver what their customers demand (or do not demand). If customers demand better quality, they'll get it.

The solution is not to simply give up on application quality, but rather to build quality control checkpoints into the application development process just like manufacturing organizations have been doing for years. Quality assurance policies generally consist of broad statements that indicate an organization's commitment to quality, but they really don't contribute to specific quality management. Quality control policies, on the other hand, consist of very specific checkpoints and objective metrics against which software is tested at intermediate points within a project.

Manufacturing organizations check quality early and often in their processes. This allows factories to move products through the line quickly and efficiently, while ensuring reasonably high quality standards. A software development team is similar to a manufacturing team in that each member produces a component for the "product," software package or system. Manufacturing or software development team members must check their own "product" at the end of each development phase. The finished product should then be checked again at the end of the manufacturing or development process to see how well all of the individual pieces and team members fit and work together.

This may seem like a simple an-alogy, but the concept is important. Consider how many software teams you've seen that appear to be completely disjointed. I've been in many project team meetings where even though the team consisted of only five or six people, none of the team members had any idea what the others were doing. Expand team sizes to fit large e-commerce projects, and chaos can quickly ensue, producing lousy application quality. A tight, simple, but elegant quality control (QC) process helps team members meet quality standards, communicate better and produce a better product.

Quality control checkpoints should be set at the end of each development phase or at the completion of any milestone. The major phases of a software development project generally include planning, discovery, requirements analysis, design, development, delivery and project review. On large projects, the requirements analysis, design, development and delivery phases as a whole may contain too many deliverables or may be too long. In those situations, QC checkpoints should be built in where appropriate. One commonly used rule of thumb is to create QC checkpoints at the end of each 40- or 80-hour work unit.

A QC checkpoint consists of a defect and metric examination by the project team, a user or customer review, and a revision to the remaining plan for that work unit. Some sample defects and metrics for each major phase include the following:

• Planning—time and budget accuracy, appropriate project participant inclusions/exclusions;
• Discovery—work units identified/ forgotten, organizational units identified/forgotten, data gathered is more complete and accurate;
• Requirements analysis—defects in business and/or data flow diagrams, time to sign-off requirements;
• Design—workflow defects, walkthrough iterations, design accuracy as measured through required revisions to requirements documents;
• Development—program bugs, required testing iterations, programming accuracy as measured through required revisions to design documents, programming accuracy as measured through required revisions to requirements documents;
• Delivery—bug counts, software-caused system downtime, cutover costs;
• Project review—overall project plan accuracy, lessons learned.

There are certainly many other metrics and defect examinations that can be incorporated into phase, milestone and project reviews. The primary goals are to ensure that there are always specific, measurable objectives applied to each QC checkpoint, and that the QC checkpoints are incorporated at the right time.

There will be a lot of organizational resistance to applying manufacturing-style QC controls to software development. Unfortunately, developers and project managers have developed a "good enough" mentality. They've managed to brainwash users into believing that "good enough" is all that is possible. Yet manufacturing companies, bridge engineers, doctors and airline pilots seem to do a pretty good job of quality control every day. The key to incorporating QC techniques into an organization is to build demand at the end-user or customer level. As e-commerce proliferates, organizations and their IT staffs will find that an increasing percentage of their customers are external to the organization. This means e-commerce application quality must improve.

Customers who purchase products, services or information over the Web demand high-quality applications that zoom. The average Web surfer won't take kindly to multiple "Page Not Found" or "Data Retrieval Error" messages. Performance matters, too. Both business-to-business and business-to-consumer Web users shop on the Internet because they want the experience to be better, faster and cheaper than traditional purchasing. Making someone wait three minutes for a simple purchase confirmation will lead to high shopping-cart abandonment rates. The bottom line is that "good enough" isn't for most Web customers. Organizations will quickly find that competitors who push for maximum quality win in their respective marketplaces.