Nokia Test – By Jeff Sutherland (Notes taken from this video)
Everyone wants to know “How well are we doing and how do we get better?”
9 questions
- Iterations – Time-boxing – can a team execute a time-box 4 weeks or less? In Scrum, we call it a sprint – and are those sprints stable? The same length over and over again? If the team can do that, yes or no, that’s what we want to know.
- Testing – tested at the feature level at the end of a sprint so a user can use it and give feedback – that is what causes the implementation of software to accelerate and have the software do exactly what the user wants. We find that delayed testing causes an extreme amount of extra work – testing now vs three weeks later is one hour vs 24 hours – i.e. 2 years to deliver sth we can do in a month! And the answer is Yes! So, the second question is “do you have the features tested at the end of every sprint? Yes or no?
- Specifications – Is there enough information provided by the Product Owner so that the team understands what they need to do? The pieces are small enough so that the team can easily absorb it, the team can estimate it, they know how to test it, all of these things – Yes or no?
- Product Owner – in Scrum, the product owner is responsible for creating the product backlog for the team to implement, ordering that backlog and having one backlog that the team is responsible for executing – do they have a product owner that’s able to deliver good backlog? Yes or no?
- The Backlog – it’s really important that the team has seen the backlog and the team that is going to work on it has estimated the size of the backlog before they go into sprint planning – if they don’t they get into the planning meeting right at the beginning of the sprint and there’s lots of questions, some of them the product owner can’t answer, so it drags out the sprint planning meeting and it gets them off to a bad start – so, do you have product backlog that is in a ready state? It’s been looked at, it’s been estimated, all the questions are answered before sprint planning starts, and then in sprint planning, you can look at it and figure out how much you should take? Yes or no?
- Estimation Process – usually done before sprint planning in terms of size – relative sizing – if a team uses planning poker, it’s based on a Wideband Delphi formal technique invented for the department of defence https://en.wikipedia.org/wiki/Wideband_delphi – if they use that, we find that teams can often estimate forty times as fast as traditional estimation procedures – it can cut the whole project planning cost by 80% by doing this right – Is the team using planning poker to estimate the backlog? Yes or no?
- Burndown Chart – Does the team have a burndown chart that shows how much work they have left until the end of the sprint and are they updating that on a regular basis? Yes or no?
- Disruption – We want the teams to take backlog and have that backlog stable during a sprint – is there anybody coming into the team, changing their priorities, pulling people from the team, or generally causing mayhem? It’s surprising how many teams have this problem. So either you don’t have disruption, or you do, yes or no?
- Team – is the team able to operate in a self-organised state? Does the team pull the backlog? Do they decide themselves how they are going to work the backlog? Can they operate without someone assigning them tasks and telling them what to do all the time? Yes or no?
When we start with new teams, even when they’ve been doing scrum for a while, we find that they usually score about 40%. As we work to improve these scores, if the score goes up to about 60% they usually about double their output. And if it goes beyond 80% they often triple output. We have had teams within Openview Venture Partners that have tripled output in the first two or three sprints just by looking at this test.
Notes taken from Jeff Sutherland’s video above which is at
https://www.scruminc.com/scrumlab-open/
and also at https://youtu.be/1yZ3J8C4MK0
For further reading see:
https://www.scruminc.com/nokia-test-where-did-it-come-from/
Download pdf of powerpoint with instructions, advanced topics-Nokia Test
http://openviewpartners.com/scrum-creator-sutherland-joins-openview/#.WQbmeIgrI2w
