One of the most popular reports people use to showcase failure of software development is the Standish’s chaos report .
In 1994, Standish reported a shocking 16 percent project success rate, another 53 percent of the projects were challenged,and 31 percent failed outright. Even though the new reports from Standish show better numbers still they cause a lot of heartburn to software companies and investors.
However recently J. Laurenz Eveleens and Chris Verhoef have published a paper in IEEE challenging the numbers stated by Standish report. The major problems in the Standish report seems to be around the way numbers for the Successful and Challenged projects are gathered. According to Eveleens and Verhoef, these Standish figures are:
“misleading, one-sided, pervert the estimation practice,
and result in meaningless figures”
The problem seems to be coming from the way the Successful and Challenged projects are defined. The definition seems to have loop holes due to which many valid projects are not considered leading to wrong result.
You can read the entire IEEE report here 
6 comments:
Hi Venkatesh,
Here's a newer version of the Standish Chaos Report, you will see that the success percentage has dramatically improved (though far from perfect) from 1994 till 2009.
Thanks PMHut for sharing the newer version.
Great point. The numbers in the Standish report have been consistently questionsed by Robert Glass as far back as 2005. One of his articles is available here - http://portal.acm.org/citation.cfm?id=1145301
Venkatesh,
I thought you might be interested in an interview I recently did with Chris Verhoef, one of the two lead researchers that wrote "The Rise and Fall of the chaos report".
Chris provides additional background information on his research.
Here is a link:
http://www.guerrillaprojectmanagement.com/the-chaos-report-myth-busters
Thank you.
Venkatesh
Thank you for sharing this.
I just wanted to share that I did an interview Mr. Chris Verhoef of the University of Amsterdam. I think you will find interesting insights and background information about the report he produced with his team.
The post is called: The “Chaos Report” Myth Busters
Here is the link:
http://www.guerrillaprojectmanagement.com/the-chaos-report-myth-busters
You can also find my own view on the Chaos Report findings (that started the conversation with Mr. Verhoef) in this post titled: Let’s say “No” to groupthink and stop quoting the Chaos Report
Here is the link:
http://www.guerrillaprojectmanagement.com/let%E2%80%99s-say-no-to-groupthink-and-stop-quoting-the-chaos-report
I would love to hear you thoughts.
Thank you in advance.
Samad Aidane
Hi All,
The issue is not whether the later version reflects an improved success percentage - the larger point is that, are there logical/mathematical holes in the argument put across by J. Laurenz Eveleens and Chris Verhoef - including the content in the full paper. The disclaimer by Standish group a'int good enough.
Ashok
Post a Comment