The Agile Architect

How Agile Are You? Let's Actually Measure It! (Part 5: Product Ownership)

Our Agile Architect shares the fifth part of his Agile Assessment, focusing on product ownership.

• By Mark J. Balbes, Ph.D.
• December 15, 2015

• Part 0: Introduction
• Part 1: Technical Craftsmanship
• Part 2: Quality Advocacy
• Part 3: User Experience
• Part 4: Team Dynamics
• Part 6: Project Management and More

So far, aside from the introduction in this series of articles, we've covered the agile assessment areas for technical craftsmanship, quality advocacy, user experience and team dynamics. Now we're on to the fifth area, product ownership.

About Product Ownership
Product Ownership is the driving force behind delivering value to your users. A good product owner stays in constant touch with the user community and the product stakeholders in order to balance all of the forces that drive the decision on what to build next and how. The product owner is also the primary liaison to the development team, bringing order to the chaos created from the cacophonous and dissonant voices of the many competing users and stakeholders

I have discussed Product Ownership in these previous Agile Architect articles.

• Defining Your Product with Agile Stories
• The Art of Writing Agile Stories Redux
• Measuring the Business Value of Projects with Agile
• The Agile Tailor and Other Stories
• The Agile Act of Continuous Delivery
• To Think Agile, What's the Real Question?

Below are the assessment specifics for this area. Please see the introduction article for a description of the 0-5 scoring methodology referenced. While not shown below, zero should be used when there is no capability in any particular area.

Area 5: Product Ownership Scoring

A. Identified Stakeholders (RACI)

1. There is some general idea of who the stakeholders are but it is not clearly defined nor written down. There is no idea of what level of involvement the different stakeholders have. Stakeholders wield authority based on personality.
2. The team is attempting to identify and clarify their stakeholders.
3. The team understands who their stakeholders are, what their roles are and what their levels of involvement are.
4. The team is monitoring the involvement of their stakeholders and experimenting with how they interact. They are measuring the effect of this on their metrics.
5. The team actively experiments with new ways of interacting with their stakeholders and is using metrics to measure the effects.

Principles: Value Delivery, Harnessing Change, Business and Development Collaboration, Empowering the Teams, Communication, Self-Organization

B. Stories (INVEST)

1. There is some way that work is organized, whether as stories, tasks or other units of work.
2. Rudimentary stories are created. They may be too big or not always user-focused.
3. Stories exist for all development work and meet INVEST criteria.
4. The team single-sizes stories and has their own criteria for the appropriateness of a story, e.g., acceptance criteria on the quality of each story's acceptance criteria.
5. The team actively experiments with new methodologies and practices and is using metrics to measure the effects.

Principles: Value Delivery, Harnessing Change, Frequent Delivery, Business and Development Collaboration, Empowering Teams, Communication, Measuring Progress, Sustainable Pace, Simplicity, Self-Organization

C. Acceptance Criteria

1. Stories may have some loosely defined acceptance criteria.
2. The team attempts to put acceptance criteria on every story. Acceptance criteria may be incomplete or hard to understand. The acceptance criteria may or may not follow the gherkin format.
3. The team has well defined acceptance criteria on every story. The acceptance criteria follow a common format (whether gherkin or something else). Acceptance criteria are used as the basis for user acceptance testing, whether manual or automated.
   
4. The team applies advanced concepts and is using metrics to measure their abilities.
5. The team actively experiments with new methodologies and practices and is using metrics to measure the effects.

Principles: Value Delivery, Harnessing Change, Frequent Delivery, Business and Development Collaboration, Empowering Teams, Communication, Measuring Progress, Sustainable Pace, Simplicity, Self-Organization

D. Story Sizing

1. The team has a sense of relative size based on an individual's or sub team's assessment.
2. The team is trying to apply planning poker or other mechanism to estimate relative size. This doesn't happen all the time.
3. The team breaks down stories until they are small. They use planning poker or some other objective mechanism to create relative sizes for all stories based on input from the entire team. Their process is written in their team charter or equivalent.
4. The team is reviewing their story sizes for completed stories and is using this as feedback to help them improve future estimates.
5. The team actively experiments with new ways of sizing stories and is using metrics to measure the effects.

Principles: Value Delivery, Frequent Delivery, Business and Development Collaboration, Empowering Teams, Communication, Measuring Progress, Sustainable Pace, Simplicity, Self-Organization

E. Delivering Value

1. Delivered value is not quantified. There is a qualitative sense of relative value of features.
2. The team is attempting to quantify relative business value. Relative estimates are based on the gut sense of the stakeholders.
3. The team routinely establishes relative quantitative business value based on informed qualitative evaluation.
4. The team is attempting to validate that the business value is realized once the feature is deployed. Metrics are collected and analyzed in order to support this analysis (e.g., reduction in number of calls to customer support, sales increases due to the new feature perhaps as indicated by a customer survey).
5. The team is able to estimate absolute business value (real dollars) for a feature. The team is able to tie real increased business value (increased revenue, cost reduction) to specific features and compare to estimates. The team can use previous data to help predict future business value.

Principles: Value Delivery, Business and Development Collaboration, Communication, Measuring Progress

F. Demos

1. The team occasionally demonstrates their progress to key stakeholders. These occur ad hoc because the stakeholders ask for them on occasion.
2. The team attempts regularly scheduled demos. The demos are weak with little to show. Some of the work from the week is non-demonstrable, indicating that the team is not focused on customer value. Attendance by stakeholders is sporadic.
3. The team has regularly scheduled demos with key stakeholders in regular attendance.
4. The team measures the effectiveness of their demos and the feedback that is solicited from it. The team experiments with different ways to hold demos and measures the impact.
5. The team actively experiments with new methodologies and practices and is using metrics to measure the effects.

Principles: Value Delivery, Harnessing Change, Business and Development Collaboration, Empowering the Teams, Communication, Measuring Progress, Sustainable Pace, Self-Organization

G. User Feedback

1. The team receives user feedback ad hoc. Feedback is not collected nor organized in any consistent fashion.
2. The team solicits user feedback ad hoc. They are figuring out how to collect and analyze the data received.
3. The team has a regular cadence of gathering user feedback including regularly scheduled demos and at least one way to interact with end users. Feedback is collected and analyzed in a basic way. The feedback is used to inform decisions on features being built and the new feature pipeline.
4. The team aggressively gathers user feedback including usability testing with end users. Feedback is incorporated into the processes to evaluate and prioritize features and in ideating on new features. The team quantitatively measures the effects of the feedback on the product.
5. The team actively experiments with new methodologies and practices to gather and analyze user feedback and is using metrics to measure the effects.

Principles: Value Delivery, Harnessing Change, Frequent Delivery, Business and Development Collaboration, Empowering Teams, Communication, Measuring Progress, Continuous Improvement

H. Prioritization

1. Individuals choose what they work on based on their own ideas of priority.
2. The team is attempting to prioritize work based on input from multiple stakeholders. Prioritization is mostly driven by personality or whoever has the loudest void. Some attempts at qualitative or quantitative evaluation of the work are being made.
   
3. The team evaluates work quantitatively when possible and qualitatively otherwise. The team has a stated understanding of how priorities are set and by whom.
4. The team has an identified approach to conflict resolution for prioritization. Metrics are being used to evaluate whether the prioritization is effective.
5. The team actively experiments with new methodologies and practices and is using metrics to measure the effects.

Principles: Value Delivery, Harnessing Change, Business and Development Collaboration, Empowering the Teams, Communication, Self-Organization

I. Release Cadence

1. The team seldom releases and has no repeatable process for releasing.
2. The team is in the process of putting together a repeatable release process. Timing of releases is still ad hoc.
3. The team follows a prescribed process for when releases occur (e.g. releases every week, releases when an MMF is finished, releases after X amount of business value is created). Releases are deployed mostly with automated scripts with some manual tasks. The team follows a prescribed release testing process which is mostly automated with some manual tests. Details of the release (e.g. release notes) are communicated to other members of the organization, customers and users through a manual process.
4. Releases occur on a regular basis. Releases are fully automated. The team gathers metrics on the effectiveness of the release and uses the information to improve the process for the next release. Details of the release (e.g. release notes) are communicated to other members of the organization, customers and users through a manual process.
5. The team experiments with different release methodologies and practices and is using metrics to measure the effects. The product owner collects metrics on the effectiveness of their communication with members of the organization, customers and users and uses this information to improve communication for the next release.

Principles: Value Delivery, Harnessing Change, Frequent Delivery, Measuring Progress, Sustainable Pace, Continuous Improvement

J. Backlog

1. The product owners have some notion of future work scattered among disparate data sources (e.g., e-mails, spreadsheets).
2. The product owners have the beginnings of a story backlog. Work to be done is collected in one place. Story sizing and business value are sporadically assigned.
3. The product owners have a prioritized, organized backlog. Relative assessments of level of effort and business value are assigned.
4. The product owners have a process around organizing their backlog, e.g., a Kanban system for discovery and story creation activities. Metrics such as cycle time and level of effort on story creation are tracked.
5. The team actively experiments with new methodologies and practices to groom, manage and prioritize the backlog and is using metrics to measure the effects.

Principles: Value Delivery, Harnessing Change, Business and Development Collaboration, Communication, Self-Organization

K. Product Owner's Ability To Embrace Change

1. The product owners reluctantly embrace change, e.g., changing business direction from upper management.
2. The product owners are learning to be eager to embrace change. They understand that change has a cost but that agile methodologies reduce that cost. They struggle with how to implement that change with the development team.
3. The product owners are eager to embrace change. They understand how to work with the development team to minimize the cost of change and how to redirect the development team's efforts. (This could be as simple as reprioritizing the stories in the backlog.)
4. The product owners measure the impact of change on the business value and level of effort, both theirs and the development team's.
5. The product owners actively experiment with new methodologies and practices to introduce change and are using metrics to measure the effects.

Principles: Value Delivery, Harnessing Change, Frequent Delivery, Business and Development Collaboration, Communication, Project Management

Time to move on to the final areas of this assessment.