Forecasts, Estimates and Cost Accounting
Forecasts, Estimates and Cost Accounting
#NoEstimates?
I’ve tried to avoid getting involved in the ongoing #NoEstimates debate. It seems more like a religious war than a discussion with much prospect of a useful outcome. And a classic case of the Analytic-minded folks butting heads with the Synergistic-minded (and a few Ad-hoc perspectives thrown in for extra confusion).
For me, it also seems like a non-argument. By which I mean that all the knowledge is out there, should folks only but seek to look. For myself, I have several perspectives, drawn from these bodies of knowledge, that I shall continue to apply in the context of estimating and #NoEstimates.
The Theory Of Constraints Perspective
I don’t recall much in Goldratt’s teachings about estimates, per se. But he has written much about the futility of forecasting, e.g. customer demand for products. I suggest his arguments also hold true for forecasting costs (estimating). For more info you might like to take a look at his books, and in particular “It’s Not Luck”.
The Systems Thinking Perspective
Systems Thinking has a relevance to cost estimation, in that systems thinking (c.f. Goldratt, Ackoff) observes that a system is a collection of parts, such that improving the performance of the parts of a system taken separately will negatively impact the performance of the whole. In fact, such “local” improvements can entirely destroy an organization.
Cost Accounting assumes that the cost of each part, each operation, can be known separately (“local costs”). This is a false assumption. I suggest that this means the estimation of costs can, in reality, only produce useful numbers when considered in the context of the system (organisation) as a whole.
See also: “Throughput Accounting” ~ Corbett
The Nonviolent Communications Perspective
From this perspective, we can choose to see folks’ requests for estimates as a means for meeting some of their needs. I’d suggest that some other folks see this means as sub-optimal, in that these other folks believe that there are better means for those folks to get their needs met than through estimates and estimating. And I’d also suggest that for those other folks, having to provide estimates is not meeting their needs. Which is triggering in them various negative feelings, possibly including anger, frustration, hostility and anxiety.
So, applying this knowledge, we might choose to discuss what needs all these folks have, which ones are being met and which not, and some options for effective means for getting everyone’s needs met. Hopefully this might lead to an outcome where folks can agree on a mutually joyful way forward.
The Covalent Perspective
In any non-trivial endeavour, there may be some number of different stakeholders and stakeholding communities, each with their own set of needs. These different needs can and will, at least from time to time, conflict in possibly mutually-exclusive ways. The Covalent approach recognises this and focuses on making folks’ needs explicit and visible, such that these conflict can be resolved, to the extent that is ever possible.
See also: “Competitive Engineering” ~ Tom Gilb
– Bob
Think Different
Hi Bob,
I think there’s also a psychological aspect to estimation that should be taken into consideration.
Consider, for example, chapter 23 of Kahneman’s “Thinking, Fast and Slow,” some of which is available as part of the limited Google Preview: http://books.google.co.uk/books?id=AV9x8XakdV0C&lpg=PP1&pg=PT372#v=onepage&q&f=false
The chapter opens with an anecdote about how estimating the work needed to complete a text book and considering the risks helped him to take an more dispassionate “outside” view and identify a problem.
At the end of the first year they used two key estimation methods: task decomposition and comparison to similar projects.
Listing all the chapters and estimating each one told them that it would take seven years to complete the whole book.
Their publisher told them that 40% of similar endeavours failed to deliver.
A problem he then chose to ignore.
“We should have quit that day. None of us was willing to invest six more years of work in a project with a 40% chance of failure. Although we must have sensed that persevering was not reasonable, the warning did not provide an immediately compelling reason to quit. After a few minutes of desultory debate, we gathered ourselves together and carried on as if nothing happened. The book was eventually completed eight(!) years later… The initial enthusiasm for the idea in the Ministry of Education had waned by the time the text was delivered and it was never used.”
Where is the joy in working so hard to deliver something that nobody uses?
Would they have been better off avoiding the estimation activities and enjoy blissful ignorance?
See also http://en.wikipedia.org/wiki/Planning_fallacy
Wouldn’t the joy have been saving a years worth of effort by doing the estimate sooner?
Bob, Goldrate’s domain was manufacturing. That “prediction” of product demand fits with the process. In the project development world where a needed capability, the to Space Station, stay 6 months, return home, or process insurance transactions for $0.07 each versus our current $0.11 the demand is defined in measures of effectiveness and measures of performance. The details – the technical and operational requirements and their implementation are emergent in many cases.
The notion of not estimating the capacity for work, the production of work and the cost for that work would cause Goldrate to smile. For the queuing process in that paradigm, we need to know the arrival rate and the throughput rate. Buffers and Ropes protect the production capacity from overload. Not knowing the capacity for work and in the end the actual cost for providing that capacity is not Goldrate, it’s bad management
The #NE paradigm has yet to come in contact with the reality of spending other peoples money. It seems to conjecture that their is somehow a benefit to no knowing the cost of the work or the duration of the effort of that work.
The “small slices” approach is workable – assuming the work can in fact be sliced in equal portions. With knowledge of the capacity for work (productivity) and equal sized small slices a throughput can be defined. But that still ignores the need to estimate that capacity before starting work. Goldrate is a production system, where the cost or materials, cost of production, and value of the produced product are known to some level of confidence “before” starting.
Bob, care is needed in referencing The Planning Fallacy and Kahneman and Tversky paradigm. They did not suggest that planning has no benefit. They suggest, and we use continuous probabilistic planning with confidence levels. The challenge is to development credible confidence intervals. at 20% probability of finishing on or before a need date is a “planning fallacy.” The anchoring and adjustment processes described by Kahneman and Tversky have direct solutions in the probabilistic forecasting paradigm – Reference Class Forecasting is the start.
This approach is used – mandated actually – in the domain we work, which also includes software intensive systems. Those not familiar with statistical processes around Monte Carlo Simulation using Reference Class Forecasting may find it useful. NASA, DOD, DOE all base their cost, schedule, and technical performance estimating and control system on Kahneman and Tversky’s principles as do Oil & Gas exploration firms.
Ged,
Domain and context are context are critical to the estimating discussion. I work in a domain where software intensive systems are developed and used to control “things.” Manned and deep space flight, processes, power system. Reference Class Forecasting is used to “estimate” cost, schedule, and technical performance. Rarely is something completely new that no reference class cannot be found or developed.
Pingback: Five Blogs – 20 August 2013 | 5blogs
I’m used to abstract discussion but this one has so little concrete connection to my set of realities that I’d suggest that some stories be told or linked to so that I, or anyone, might have a chance to make some useful suggestion.
What’s the problem? Who cares? How could a proposal get evaluated before or after testing?
From decades of following Gilb’s writings, I’ve developed a strong appreciation for using quick guestimates followed by brief, cheap tests. Both of these come after figuring out who cares, i.e. stakeholders, what the top goals are, i.e. the problem to be solved, and, critically, how we shall make numbers that will represent progress toward those goals. (Without numbers, we are merely making feel-good games that will never matter much.)
Identifying and gathering stakeholders to suck opinions from is the least clear part to me. Goal gathering and progress measurement can be compressed into an hour or two each, especially with experienced coaches involved. The testing might take a minute, an hour, a day, or even a week. Otherwise invent a smaller test,
Still, I’d love to watch an hour of video collected from an actual successful project in virtually any realm. Wouldn’t you?