The Agile Architect

How Agile Are You? Let's Actually Measure It! (Part 2: Quality Advocacy)

Our Agile Architect shares the second part of his Agile Assessment, focusing on quality advocacy.

• By Mark J. Balbes, Ph.D.
• December 15, 2015

• Part 0: Introduction
• Part 1: Technical Craftsmanship
• Part 3: User Experience
• Part 4: Team Dynamics
• Part 5: Product Ownership
• Part 6: Project Management and More

We started this series by looking at the basics of assessment, and then looked into the first assessment area, technical craftsmanship. In this section, we will dive deeper into the second area of quality advocacy.

About Quality Advocacy
Quality advocacy is the act of taking action to ensure that an agile team follows quality process and practices throughout everything that it does. Unlike quality control which ensures that, after the thing is built that it was built right, and unlike quality assurance that gives responsibility of quality processes to specific team  members, quality advocacy is a whole-team approach to quality.

I have discussed quality in these previous Agile Architect articles:

• Project Management Is Not Enough: 4 Crucial Agile Techniques
• Dealing with Technical Debt: An Agile Horror Story
• Agile Deadline Ahead: Calculating Velocity
• Good Agile Is Messy Agile
• True Quality Assurance in an Agile World
• Agile Expressionism 101
• To Be Agile, Beware the Comforts of Ritual
• In Agile, Simple Is Not...
• Finding the 'Simple' in Agile
• Agile Testing Heresy: Are You Testing Too Much?
• Making Agile Compromises
• Getting Things Done the Agile Way

Area 2: Quality Advocacy Scoring

A. Focus on Code Quality

1. Code quality is enforced haphazardly through manual code inspection. No explicit code quality standards exist.
2. Some code quality standards are defined. Compliance is monitored through a manual but consistent process.
3. Comprehensive code quality standards are defined. Compliance is enforced through a combination of manual and automated processes.
4. Comprehensive code quality standards are defined. Compliance is enforced through fully automated processes.
5. The team actively experiments with new methodologies and practices to increase code quality and is using metrics to measure the effects.

Principles: Harnessing Change, Frequent Delivery, Sustainable Pace, Technical Excellence, Simplicity

B. Process Definition (Is the Process Visible, Obvious, Simple?)

1. Some passing knowledge from reading books and articles. Not yet applying the knowledge.
2. Actively trying to apply knowledge from training, books and coaching
3. Team understands the practice and uses it regularly. They are measuring their progress/capability, etc.
4. Team applies advanced concepts and is using metrics to measure their abilities.
5. Team actively experiments with new methodologies and practices and is using metrics to measure the effects.
   

Principles: Value Delivery, Harnessing Change, Frequent Delivery, Business and Development Collaboration, Empowering the Teams, Communication, Measuring Progress, Sustainable Pace, Self-Organization, Continuous Improvement

C. Internal Quality

1. Internal quality is maintained by the technical expertise of the individual developers.
2. Internal quality is maintained by automated tests and code reviews. Code refactoring may happen sporadically.
3. Internal quality is maintained by a consistent application of test-driven development, code reviews (optional) and constant refactoring. New bugs are triaged and generally given priority over features. The team is practicing continuous integration either manually or through a CI server.
4. Team uses metrics to continuously monitor and track internal quality. This might include number of defects found per week, velocity, code complexity, adherence to coding standards and good design principles.
5. Team continues to experiment with ways to increase the level of internal quality and track how it affects their metrics including their ability to deliver solutions quickly.

Principles: Value Delivery, Harnessing Change, Frequent Delivery, Sustainable Pace, Technical Excellence, Simplicity, Continuous Improvement

D. External Quality

1. Team tracks bugs found by users in the production app.
2. Team has a manual process to test the system before a release.
   
3. Team has a defined approach to measuring and ensuring external quality. This includes a test plan (manual or automated), functional testing, exploratory testing and user acceptance testing.
4. The team manages and continues to expand a comprehensive suite of automated tests including unit, integration, functional, non-functional and user acceptance tests. The team tests that the app adheres to all design and usability guidelines. Most manual testing is exploratory in nature.
5. Team actively experiments with new methodologies and practices and is using metrics to measure the effects.

Principles: Value Delivery, Frequent Delivery, Business and Development Collaboration, Sustainable Pace, Technical Excellence, Simplicity, Continuous Improvement

E. Team Ownership of Quality

1. Team maintains quality through the individual initiatives of the team members
2. Team has basic processes in place to ensure quality. The acceptable quality level is maintained through tribal knowledge.
3. The team has a well defined quality level that they adhere to. Quality is managed and monitored through a combination of defined processes, tribal knowledge and team diligence.
4. Team quality processes and practices are completely encoded into their tools, behavior, and working environment. Quality is measured and tracked.
5. Team actively experiments with new methodologies and practices and is using metrics to measure the effects.

Principles: Value Delivery, Harnessing Change, Frequent Delivery, Empowering the Teams, Sustainable Pace, Technical Excellence, Simplicity, Self-Organization, Continuous Improvement

F. Defect Management/Measurement

1. Defects are fixed ad hoc. No formal tracking of known defects.
2. Defects are tracked as they are discovered. Defects are scheduled to be fixed.
3. Defects are tracked and managed either electronically or manually. Defects are triaged and regularly reviewed. Defects are scheduled to be fixed.
4. Defect rates are measured and fed back to the team. The team charter addresses how defects will be managed and fixed.
5. The team actively experiments with new methodologies and practices and is using metrics to measure the effects.

Principles: Value Delivery, Harnessing Change, Frequent Delivery, Measuring Progress, Sustainable Pace, Technical Excellence, Simplicity, Continuous Improvement

G. User Acceptance Testing

1. Ad hoc manual user acceptance testing.
2. Managed manual user acceptance testing.
3. The team manages a combination of manual and automated user acceptance tests. The process and/or charter defines when and how UATs are created and maintained. UATs are part of the CI process, whether manually or through a CI server.
4. Automated user acceptance testing is an integral part of the team process and practices. The team measures and monitors failing UATs and incorporates feed back to improve their processes.
5. Team actively experiments with new methodologies and practices and is using metrics to measure the effects.

Principles: Value Delivery, Business and Development Collaboration, Communication, Measuring Progress, Technical Excellence

H. Exploratory Testing

1. The team performs ad hoc manual exploratory testing.
2. The team performs regular manual exploratory testing maintained through tribal knowledge.
3. The team incorporates exploratory testing into their standard practices and processes as expressed in the team charter or other agreement. Results of exploratory tests are used to drive creation of new automated regression tests
4. Exploratory testing is an integral part of the team process and practices. Classes of exploratory tests are maintained through documentation or other means to ensure that an appropriate level of exploratory testing is performed. The team measures and monitors failing exploratory tests and incorporates feed back to improve their processes.
5. Team actively experiments with new methodologies and practices and is using metrics to measure the effects.

Principles: Value Delivery, Technical Excellence, Continuous Improvement

Please see the next article in this series to move on to the next assessment area, user experience.