EN 
DE Meaningful estimates even with limited information
Author: Andreas Stucki, Solcept AG
Contribution – Embedded Software Engineering Congress 2017
Estimating effort is always easy when you've done something before. But what if everything is new? What if information about the project is scarce? This article presents various estimation methods, both well-known and lesser-known, along with soft factors to consider. It then demonstrates how to combine these methods for two scenarios: first, for new but reasonably well-defined projects, and then for projects that are still more like ideas.
Is estimation really that important?
This question depends heavily on whom you ask in your organization. For us as an engineering firm, accurate estimation is vital for our survival, as even miscalculations of a few tens of percent can be existential threats.
What is to be valued?
The following article demonstrates how to estimate development effort in two scenarios. The first scenario involves a development process where the specifications and, more importantly, the steps toward the product are reasonably well-defined. The second scenario involves a product manager presenting a product idea on a single A4 page and demanding a cost estimate – "by the end of the week, please.".
Preparation
Especially when little information is available, it's important to have access to all available information. Therefore, it's worthwhile to have a conversation with, for example, the author of the specification, to clarify what is implied. This information can then be incorporated into a WBS (Work Breakdown Structure, also known as PSP: Project Structure Plan) as shown in Figure 2 (see below). PDF) translate this into a hierarchical breakdown of the project into increasingly smaller work packages, ideally presented in a table. It makes sense to structure these packages hierarchically, ideally along a project lifecycle. That is, along the steps of "how we develop such things here".
But what do we do with an idea on an A4 page? A WBS would require us to create an architecture of the product. Here, we can use an FBS (Functional Breakdown Structure) as shown in Figure 3 (see below). PDFCreate a comprehensive document containing all functions, ideally in a hierarchical order. It's especially important to separate the functions for different development groups, such as software and hardware developers. This allows each group to value its own specific needs.
This list should then be reviewed with all involved parties. This way, you can identify items that are "obviously essential." And by the way: if you consistently create a similar WBS (Work Breakdown Structure), a template will eventually emerge, ensuring that no steps are overlooked.
What do you estimate?
Ultimately, it always comes down to money; the hourly rate is fixed, so you can't avoid estimating the effort in hours. Especially with a Functional Business System (FBS), where many things are still unclear, you can simplify things by using categories. This means roughly sorting the functions by complexity into, for example, S, M, L, XL, and then estimating the effort for each category.
How to estimate?
The simplest method is the heroic approach: you give the list to the guru, who "should now say what the effort is.".
Because this often doesn't work, the Rand Corporation invented Wideband Delphi [RC], revived as Planning Poker [JG]. In this method, each expert submits their estimate for each work package, but they only learn about everyone else's estimates after they have submitted their own (details of the procedure: [PP]). The consensus achieved through the iterative application of this process is usually very good, as no one can simply follow the lead author.
Parameter-based estimation allows for simple yet accurate estimation. For example, if you know how much time is spent per weekly project meeting, multiplying this by the number of attendees and the number of weeks provides a good estimate of the effort required.
Historical data also provides good clues. For example, we know that a project typically requires roughly 10 times more effort than the cleanly completed system design phase. So, if this phase takes 400 hours, the entire project will likely require approximately 4000 hours. Note that this relationship can be non-linear; for example, for a large project, the effort might only need to be multiplied by 8.
And you first need the historical data. This means that project controlling must take place, which records these parameters.
How do you deal with blurriness?
Every estimate is subject to uncertainty, and it is helpful to be aware of this.
In [RS] p. 112, a 3-point estimation can be found. First, the most probable effort N, the smallest possible K, and the largest possible G are estimated. The resulting expected estimate S (based on the beta distribution) is then: S := (K + 4N + G) / 6. In the same place, a modified formula can also be found, which shifts a weight from N to G to compensate for the "estimators' tendency towards optimism": S := (K + 3N + 2G) / 6.
In [WR] p. 146, this becomes a simpler formula by omitting the smallest possible estimate altogether, i.e., a 2-point estimate is made: S := (0K + 4N + 2G) / 6 = (2N + G) / 3. This saves the effort of an estimate and incorporates the compensation for optimism. The formula also has a simple justification in the normal distribution: If one assumes that G lies at 3 standard deviations, then S lies at 1 standard deviation. That is, in 84% of cases, the effort is less than S.
For a statistically more robust two-point estimation, the log-normal distribution can be used. This has the advantage of being defined from 0 (efforts below 0 are unrealistic) to infinity and can also be determined from the two parameters N and G. Its disadvantage is that some numerical analysis is required to, for example, sum the distribution.
What about the risks?
„If a project has no risks, don’t do it“ [dML].
Which risks are included in the estimate? Which are not? This is decided by the estimation team during discussion (e.g., during Planning Poker). This decision is made in the form of assumptions and remaining risks outside the margin of error. It is therefore recommended to document both the WBS and FBS for each point in the discussion to ensure they are not overlooked.
Both can then be used to assess the project's risk, since risks are assumptions that may not materialize. Quantifying these risks in terms of potential damage and probability of occurrence yields the total risk, which is a crucial component of the assessment.
What else do you need to consider?
Avoid estimating tasks that are too large: typically, tasks estimated at one time should not exceed 40 hours, with a maximum of 80 hours.
Budgeting the estimation: consider how much time you want to spend on the entire estimation process. Then you can limit the time for each estimate, for example with a timer.
Effort (hours) is not the same as duration (hours): depending on the organization and experience, developers work a maximum of 70-80% for the project, so plan accordingly.
And by the way: In principle, these methods can also be used to estimate other quantities, such as costs or storage requirements.
How does one estimate based on a specification?
If we have a sufficiently detailed specification, we proceed as follows: First, the project manager creates a WBS table based on a template, with a description of the individual work packages.
The team then conducts Planning Poker under the project manager's leadership. The team first estimates the most likely, normal scenario and then the spread, i.e., the difference to the maximum possible effort ("worst case"). The project manager documents the discussion (description, assumptions, and remaining risks) during the session.
The estimated effort for each WBS task is calculated according to S := (2N + G)/3. For a realistic estimate, the weighted risks are added to the total. A template for a spreadsheet for this estimate can be found under [Sc1]. We have developed a tool (from which the images are taken) that allows for the easy inclusion of parametric estimates based on historical data.
And what about a project idea?
And how do you estimate the cost of a project idea? The project manager creates an FBS table, describes the functions, and sorts them by complexity into S..XL. See Figure 3 in the... PDF.
The team randomly selects three functions of each complexity (S..XL). For these, they estimate N and G using Planning Poker as described above. Then, based on historical data, the team estimates parameters such as the proportion of testing and debugging in the development. From this, the distribution for the entire project can be calculated numerically or using Monte Carlo simulation. Our tool can also do this; you can find our tool for estimating control projects under [Sc2]. It also makes sense to include the sum of the risk assessment in this result.
For the estimator, this method has the advantage of allowing them to specify a range from the distribution, e.g., the 90% confidence interval. This gives their "client" a clear indication of how much risk they are willing to accept. Once the project becomes clearer, a WBS can be created instead of the FBS, allowing for a more accurate estimation of at least the next phase.
References
[RC]: Wikipedia: „Delphi Method“, 2017, https://en.wikipedia.org/wiki/Delphi_method (download: 2017-09)
[JG]: JW Grenning: „Planning Poker“, 2002, https://wingman-sw.com/papers/PlanningPoker-v1.1.pdf (download: 2017-09)
[PP]: Solcept: „Planning Poker Guide“, 2017, https://www.solcept.ch/de/pp (download: 2017-09)
[RS]: RD Stutzke: „Estimating Software Intensive Systems“, 2005
[WR]: W. Reiter: „The Naked Truth About Project Management“, 2003
[dML]: T. DeMarco, T. Lister: „Waltzing with Bears“, 2003
[Sc1]: Solcept: „ESE Congress References“, 2107, https://www.solcept.ch/de/blog/ese-kongress-2017/ (download: 2107-12)
[Sc2]: Solcept: „OnlineEstimation“, 2017, https://www.solcept.ch/de/verbrauchsschaetzung/ (download: 2107-12)
author
Andreas Stucki, Dipl. El.-Ing. ETHZ, Managing Director of Solcept AG, has 20 years of experience in project management of embedded software and hardware projects in industrial and commercial environments.
Contact
Internet: www.solcept.ch
Email: a.stucki@solcept.ch
Software Engineering Management – our training courses & coaching sessions
Do you want to bring yourself up to date with the latest technology?
Then find out more here MircoConsult offers training courses/seminars/workshops and individual coaching on the topic of Software Engineering Management / process, project and product management.
Training & coaching on the other topics in our portfolio can be found here. here.
Software Engineering Management – Expertise
Valuable expertise in software engineering management / process, project and product management is available. here Available for you to download free of charge.
You can find expertise on other topics in our portfolio here. here.
