Inspired by Gunther’s Scrum and….., I want to illustrate the model “Scrum And …..”.

Your team is practicing the Scrum framework, great. But are you making the most out of it by using more meaningful measurements? Help make meaningful changes by using measurements that reveal the problems at the team level and organizational level.

Scrum and…... No Measurement 

Technically, you are practicing Scrum if you are delivering at least one increment per sprint. But how valuable is that increment? Are you able to use it to improve your team’s performance? Measurements are important because, without data, you can only rely on your gut feeling, which may not be completely accurate. 

Scrum and…… Velocity

Velocity is a key metric in Scrum. If you’re using it, that’s step one! By measuring your velocity, you can compare the number of story points that was forecasted during the sprint planning versus what actually delivered after the sprint. This can help to recalibrate your scope and time to be more accurate. The downside is that its emphasis on time and output may encourage technical debt where accuracy and quality may be sacrificed in the face of efficiency and time. Worse case scenario, a team member may skew the points for the Product Backlog Item (PBI) just to meet the numbers. Nevertheless, velocity is a useful tool to improve your team’s performance. It is just not enough. Since it only measures output, users cannot take into account the complexity where higher output does not necessarily translate into a higher outcome. Even when you produce more features, it does not mean that users will definitely like it or increase revenues. 

Scrum and …… Outcome

This means that measuring by outcome would be better, right? When we talk about outcome, we refer to the way customers are satisfied with their product or service, whether it generates revenue for the company, or whether we can have frequent releases without any issues. The difficulty in calculating outcome is that it takes time. They become a lagging indicator, meaning that by the time we are able to collect consequential data, it may be too late to influence the result. The long cycle of feedback makes it difficult to inspect and adapt. That is why, even though measuring outcome may be exactly what you want to find out, it may not be enough to identify problems and to make helpful changes. 

Scrum and …… Evidence-Based Management (EBM)

Take the next step and adopt Scrum.org’s Evidence-Based Management. Track either at the company level, portfolio level, or product level with both outcome and output. Here are some examples below:

Outcome/Lagging Indicators 

• Current Value: Revenue, Growth Rate, Customer Satisfaction, Customer Retention, Referral  

• Time-To-Market: Release Stabilization, MTTR (Mean Time to Repair), Service Uptime

• Ability to Innovate: Innovation Index, Usage Index

• Unrealized Value: Market Share Trends, Overall Market Growth

Instead of just velocity, we can also keep track of leading indicators, such as:

• Current Value: Employee Satisfaction, Usage index  

• Time-to-Market: How many projects/products the team is working on, Lead Time, Cycle Time, Release Frequency, Work Item Age, Work in Progress (WIP) at team or portfolio level

• Ability to Innovate: Production Incident Trend, Technical Debt, Downtime Trends, Time spent on merging/resolving branch conflicts, 

• Unrealized Value: Competitor Strength/Weakness, Customer Acquisition/Defection, Satisfaction Gap Trends, Design Thinking or User Research Findings 

Check how your team is doing from different angles. Identify the problems quickly and make changes effectively.

Scrum and …... Executive Action

Now that you have more meaningful methods of measurement, you have to focus on what you do with those measurements. Scrum can be used at all levels. Teams like the product owners and development teams can take quick actions and improve on the team level. However, some impediments that affect agility, such as policy, tools, cloud, team structure, budget, career path, or appraisal model, may stem at the organizational level. It is then the Agile Leader’s role to take action and work with the Scrum Master to manage the issue. 

Scrum is useful because it helps to reveal problems. But it is up to you to do something about it. For example, one of our clients had teams in Hong Kong, Singapore, and across the whole Asia Pacific region. Each team had its own products which were being released according to its location. One day, the company’s management decided to change the team structure. The Hong Kong team was no longer responsible for doing code reviews and deployments. Instead, they had to send their codes to Singapore first. Before the structural change, the team in Hong Kong had adopted Scrum with Kanban, and they found that the Hong Kong team was able to deliver 85% of the PBI within 7 days. After the change, it took them 14 days to achieve 85% of the PBI since they had to wait for the Singapore team before proceeding. After revealing the data, it was up to the company’s management team to decide whether they wanted to take action to retain its structural change, meaning longer times, or revert to the original proceeding. In other words, Scrum is a great resource and tool for companies to utilize but it is the company’s executive actions, using the data that can make long-haul changes, that will ultimately benefit the company.