Estimation related stuff (books, tools, etc) that we have found useful:
- Preliminary guidelines for software estimation (Link not currently supported.)
- Software Estimation: Demystifying the Black Art (Best Practices) by Steve McConnell
- Agile Estimating and Planning by Mike Cohn
- The BEST library of estimation papers
- Forecasting Principles
- Paul Coombâ€™s book on IT project estimation
- Online Planning Poker
- Software Metrics: Measurement for Software Process Improvement by Barbara Kitchenham
- The COCOMO homepage
- International Software Benchmarking Standards Group, Data Repository (historical data)
Software development effort estimation papers frequently appear in academic journals, conferences and magazines such as:
- Transactions of Software Engineering
- Information and Software Technology
- Journal of System and Software
- Empirical Software Engineering
- IEEE Software
- Communications of the ACM
- Software Quality Journal
- Information and Management
- Evaluation and Assessment in Software Engineering (EASE)
- International Symposium on Empirical Software Engineering and Measurement (ESEM)
- The Australian Software Engineering Conference (ASWEC)
- International Conference on Software Engineering (ICSE)
- Agile Conference
Measuring and analyzing estimation error are the basis of estimation learning related activities, such as deciding whether or not an organization has an estimation problem, identifying risk factors related to project performance in software development, and, evaluating and improving estimation and uncertainty assessment methods and tools.
The most commonly used measure of estimation error is the Magnitude of Relative Error (MRE). MRE is identical to the measure called FASE in other branches of forecasting research. The mean MRE (MMRE) is often used to average estimation error for multiple observations. It is not unproblematic to use MMRE as a measure of estimation accuracy, and several other measures, such as PRED and MER, is sometimes used. However, all estimation error measures have shortcomings. Hence, the measure that should be used in any given case depends on the context
Many factors can affect the measured estimation error. Measuring estimation error without a clear understanding of which factors contributed most to the estimation error, e.g., without an understanding of whether a high estimation error is caused by the factor â€œlow estimation abilityâ€ or â€œhigh estimation complexityâ€, is rarely meaningful.
Papers that report estimation surveys. (Links not currently supported.)
Papers on estimation error measures and analysis. (Links not currently supported.)
There are two different sources that cause uncertainty regarding effort usage in software development projects: (i) The inherent uncertainty regarding cost usage in a software project, and (ii) The uncertainty caused by lack of knowledge about the project. However, uncertainty can be reduced by flexibility of outcome and software development process.
Accurate assessment of the uncertainty of software development effort estimates is an important part of software estimation, e.g., when deciding whether or not to embark up a project, to support the bidding process, to support decisions about how large the projectâ€™s contingency budget should be.
Clearly, the use of cost uncertainty assessments that reflect the underlying uncertainty will improve the budgeting process. Unfortunately uncertainty assessments are usually over-confident about the accuracy of cost estimates.
Papers on uncertainty assessments. (Link not currently supported.)
A plethora of estimation methods exists and there are several schemas for classifying them. Here we classify estimation methods in "expert estimation" and "formal estimation models".
- "expert estimation" is used as a label for estimation methods in which a significant part of the estimation process (particularly the final step, i.e., the â€œquantification stepâ€) is based on intuition. Expert judgment-based estimation is not a single estimation method, but a spectrum of judgment-based processes. Even within the same software development organization, processes of expert judgment-based estimation may vary from those based on pure intuition to highly structured processes that use relevant historical data.
- "formal estimation models" is used as a label for estimation methods where a substantial part of the estimation (and particularly the "quantification step") is based on the use of mechanical and analytical processes, e.g., the use of a formula derived from historical data using regression analysis.
Software effort estimation research is inconclusive regarding which estimation approach is better, e.g. a recent review of studies comparing models and experts in software development effort estimation concludes that experts typically performs no worse than the models. One reason for this may be that it seems to be difficult to develop meaningful estimation models that do not require a high degree of expert judgment as input to the models in the first place.
Papers on estimation by estimation method (Links not currently supported):
- Expert judgment
- Combination of estimates
- Function point
- Neural network
- Work break-down
- Other topics
Page 1 of 2