The Planning Fallacy

I admit: I have fallen for it more than once. As I'm sure you or anyone else who has ever had to plan a project or predict outcomes must have. So, how can we overcome it?

One of the biggest things one has to watch out for while drawing up a plan, any plan, is the budget* prediction. How much money would it take to get started? How much time until we complete? How long until the supplies arrive? Some of these questions may have straightforward answers. But in an unsure world, it’s best to temper our runaway optimism with a dose of reality. Especially in January, when the world seems rife with possibilities and we brandish new year resolutions with gusto.

*For simplicity, the budget I refer to = time + cost + personnel.

So, What is the Planning Fallacy?

It’s a phenomenon that most of us are intimately familiar with but may not know the pedantic term for. I also came across the term quite recently.

In a nutshell:

The Planning Fallacy 
= Optimism + Self-Confidence + Fervent Wish for the Best-case scenarios = Setting of Unrealistic Deadlines and Budgets.

The Planning Fallacy was identified by scientists nearly 50 years ago and has been studied thoroughly. Yet it has failed to enter the mainstream as it happens to also be one of those cautionary tales that we think do not apply to us. In a society where optimism, risk-taking, and dare-devil attitudes are praised, there is no space or time given for conscious decision making.

To counter the planning fallacy we have to ignore our intuitions, impulses and misleading external pressures.

Courtesy: Stephen Lynch’s blog.

Some Common Examples:

• Weddings: We hope and pray that our family would stick under the “budget“, but costs inevitably creep up.

• Vacation Planning: We plan within a relatively small time frame and budget, but only when we get to the vacation spot do we realize how much more there is to see or how expensive everything is.

• Home Renovation Projects: You may all know of a certain someone who starts out with a fastidious list of improvements, and then proceeds to be surprised by how many more tasks need to be added on.

When Can it Occur At Work?

Answer: At the very beginning, when we sit down to plan a project and we have only a vague idea of the requirements and goals. Often, there is pressure on managers to come up with plans quickly. Which compounds the underlying problem even more. While planning, it’s important to gather all the available information, make note of the assumptions, and keep an eye on the variables before making any predictions on the budget. But more often that not, senior management will jostle you into delivering a plan ASAP, which in turn pushes you into a situation ripe for the Planning Fallacy.

Putting aside external pressures, the other instances where I fell for the Planning Fallacy happened whenever I got too excited and carried away by a particular idea. I would be so eager to get started on the task that I would map out only the best case scenarios that would take me to my goal and come up with several creative reasons to support my hypothesis. I would push back on negative thoughts, even ones that say ‘now, hold on a minute…‘ or some other sensible course of action. I like to think that I have grown beyond that impulsive version of myself, but putting a lid on the excitement for new ideas is something I do actively work on.

How Can We Watch Out For IT?

1. Ignore your Internal Compass:

   Numerous Studies on Decision Making have shown us one thing: we humans are not good at predicting outcomes. One of the main reasons for this appears to be our inability to remember the relevant past events. Even if we have faced a similar situation before (say finishing a 500-word essay for school) we would time and time again console ourselves with an optimistic outlook. Of course, we can finish this days before the deadline. When in reality, we’ll be grinding it out the night before.
   
   We tend to remember our past in a positive light and shirk away from negative experiences. Following the same example, during my undergrad, whenever a new assignment would come around, I would remember how I had submitted my last one in on time and how it had taken me just one night to complete. I wouldn’t remember the stress of getting it done at the last minute or hunting down classmates who could help me finish. If I hadn’t scored well, I would ignore that factor as well.

   The only way to improve our internal predictive capabilities is to make an objective note of what happens each time and understand the choices we make.

2. Relying on External Data:

   AI definitely beats us at learning how to improve. The models look at the whole available data and understand what others think the answer should be and then learn to correct their own mistakes. Taking a more holistic view can help us hone our predictive powers as well.

   Take the following case: While planning for a software project concerned with bug fixes, it would be reasonable to look at previous bug fixing cycles within the same team / company and then make a prediction on the time required based on the average times taken in the past. Final adjustments can be made depending on the current scenario, but keeping an eye on the baseline appears to be the way to go. Construction projects have learnt heavily from their past mistakes and now look at other construction projects in similar environments while finalizing their estimates.

3. Involve the Entire Team in Scenario Mapping:

   Scientists involved in the study of decision making wisely also concluded that one person can never fully grasp the sum of past experiences of an entire team of people or understand the underlying technology enough to predict outcomes correctly. When I worked as a project manager, in situations lacking sufficient information I had to make guesses about my team members’ capabilities, about how long certain projects may take, the nature of the risks involved, etc.

   I found that asking for second opinions greatly improved the quality of my guesses, even though the status quo pushed me to do it on my own. We need to normalize planning together. Because, when the project does not go to plan or deadlines start slipping, the blame inevitably falls on the project planner. Executives at this time conveniently forget the screws they had tightened in the beginning.

   

   Photo by CoWomen on Unsplash

   Suggestion here:

   Sit down all the members of the team / knowledgeable stakeholders and discuss the initial plan. Even if they all are onboard with it and give you the go ahead - do not stop. Ask them to think for a few minutes and come up with worst case scenarios. How badly can this project bomb? Their inputs and observations could prove to be invaluable while revising the project roadmap!

4. Plan Small, Then Big:

   An old but ageless adage. Plan in smaller chunks then piece them together to form a coherent big picture. For instance, we tend to plan out our household chores for a given week more precisely than we would for the entire month.

5. Be Generous with your Buffer Times:

   One of the most annoying situations in the corporate world crops up when your boss demands thus: Why should Project A take so long? Does the team really need that extra week? And in those situations, I would have to breathe in and out deeply and then answer as calmly as possible. I can’t say how well that worked out.

   In fact, scientists recommend doubling planned deadlines. If I had followed that advice, I would have been laid off sooner. JK!

   In the past, I’ve had to come up with creative ways to add in buffer times: I would extend individual task deadlines by 20% which when added up gives the overall project a bigger buffer, I would add in a whole day for logistical tasks, which in the best-case scenario world may take less than half a day, and I would stubbornly add an extra week at the end of projects. It just makes sense! In the end, we are all mere humans.

6. Keep a Record:

   At the end of a project, keep a record of what went right / wrong, how off your predictions were, how correct your assumptions were, what external variables ended up having more influence than you first thought, and more importantly, what you learnt from the experience. In this case, history should not be written by the victors. Let’s keep track of where we err and work on improving our internal compasses and timekeepers however we can.

---

To Learn More about the Planning Fallacy check out the original paper published in 1977 by the Psychologists who first observed and studied the phenomenon.

Thanks for reading Fourth Waver in Tech! Let me know what your experience has been dealing with this monster of a Fallacy!