Although it is a commonly held opinion that planning poker improves the accuracy of project estimations (a small sample of which demonstrated on this question), has any defined research been done on the subject?
More specifically, I am looking for non-circumstantial information showing that planning poker would be an improvement over traditional estimation techniques.
9
Google Scholar turns up some papers
You might find the following papers useful, but they are behind a paywall and may be a little dated now:
- Combining Estimates with Planning Poker — An Empirical Study 2007
- An empirical study of using planning poker for user story estimation 2006
You might also want to consider A Case Study on Agile Estimating and Planning using Scrum 2011 (free PDF) starting around page 123.
A story from my experience.
When our team started – we were using planning poker by Mountain Goat. Since our team was distributed among 2 cities, we were short on options for estimation methods. As for the tool itself — it was Planning Poker 3.0 JIRA Add-On
Thus, as our team was growing (in size of members and in terms of experience), we found out that the planning poker does not fulfil our needs anymore. Due to the scattering factor of team members, due to issues from different projects that have to be estimated during the same meeting, due to inefficiency for estimating large piles of stories – we switched from this method to another – the Team Estimation Game by Steve Bockman.
This method worked well for us, though the preparation time for a session increased (we didn’t have a proper tool with the “online multiplayer” solution) – our scrummaster used Google Docs for preparing an estimation session.
Measurements so far based on same set of staff – (unscientific measures):
Planning poker – variance off by 24-78% from actual time it took to complete 113 tasks.
Historical, data analysis techniques off by as much as 200%. On average, however within 30% of each estimate it took to complete 113 tasks.
Commentary- I am biased I admit after the above was observed. The value i would see if there is a discussion for rationalization of each estimate. if you have a team member who thinks through carefully each step of his estimate, and also carries out his estimate within a small variance, then that person(s) would add value – however if it is a quick guess on each estimate this is a folly and a waste of time imo. completely breaking down the problem into small bits would probably be the best estimate of time expended – coupled with a long trail of historical data.
1