Why do projects fail?
It sounds like a simple question, but it has been at the forefront of the oil and gas industry for decades, the genesis of papers, panels, articles, and webinars. And, as the size and scope of projects continue to grow, this question takes on added significance. The high failure rate of projects is well documented. Companies want to find the most efficient ways of managing their investments, and when those investments do not meet their targets, they want to know the reasons.
So, why do they fail?
There is no correct answer to this. It could be a technical or procedural reason, a combination of the two, or something else entirely. Schedules may be too aggressive, goals may be unrealistic, or owners may be too slow to respond to problems.
An element of project management coming under scrutiny is the psychological culture surrounding the projects themselves. Are aggressive schedules, unrealistic goals, and slow responses symptoms of a larger ill?
This article does not attempt to explain the deeper psychological constructs of project management, like overconfidence and bias, that others have delved into (Briel et al. 2013), but it looks at how some of those constructs affect the tangible elements of a given project. How does overconfidence lead to aggressive scheduling? Does it foster a conservative attitude when problems arise? What are the best ways to mitigate these problems? Are these problems that need to be mitigated?
Aggressive Pursuit
Oil and gas is an aggressive industry. The search for valuable resources in the furthest reaches of the globe requires taking risks and capital investment, and while that search can lead to a financial windfall for companies, it is far from a guarantee that anything will come of it.
With exploration and production (E&P) projects reaching nine-, ten-, or even eleven-figure costs, companies want to ensure the best possible return on investment (ROI). Since returns can only come after a project begins producing, executives and analysts want to minimize the time to first oil as much as possible so that they can maximize the economic value of a company’s investment. This leads to more aggressive schedules, which in turn leads to trade-offs being made to meet the deadline. Important phases are given short shrift. There is less time to acquire the proper data needed for teams to make fully informed decisions.
A recent study of scheduling effectiveness reported that from 2003 to 2011, 54% of authorized E&P projects were planned with a schedule that was faster than similarly scaled projects within the industry (Nandurdikar and Kirkham 2012). Combined with an increase of 25% in the failure rate of E&P projects over the last two decades, the statistic suggests that faster schedules do more harm than good (Briel et al. 2013).
Fig. 1 shows that aggressive schedules resulted in lower than planned production. “Schedule aggressiveness,” as defined here, is the ratio of the planned execution schedule at project sanction to the industry schedule benchmark for projects of a similar nature. “Production attainment” is the ratio of actual production to the planned production at sanction. Faster schedules lead to greater chances of lower than expected production.
Briel et al. argued that projects are consistently delivering below expectations because the expectations of ROI are unrealistic. Organizations in the E&P sector often base investment decisions on overly optimistic estimates of time and cost, optimism borne of a culture that reinforces bias towards overconfidence. Excessive optimism leads companies to set an aggressive schedule for projects because it understates the mean of possible outcomes.
Stephen Cabano, president of Pathfinder, an international project management consulting firm, believes that the overconfidence also extends to a company’s ability to look at its own staff with a clear eye. In evaluating potential projects, companies rely too much on previous benchmarks and presume that their current staff has the capability to handle a large scope, even if the market has changed and the staff has little experience with a big project, he said.
“The reality of the industry is that, at least in the petrochemical world, there are huge opportunities, [and] business people are making very aggressive commitments that the project side of the business now has to achieve,” he said. “A lot of these companies have not executed projects of any significant size in the past 10 to 15 years… but now they’re being asked to manage, in some cases, projects in the hundreds of millions and billions of dollars, and they just don’t have the tools or the resources to be able to do that.”
This does not mean, though, that aggressive schedules are themselves a bad thing to strive for with projects. Richard Westney, head of the Westney Consulting Group, said that aggressive schedules can be executed properly so long as the right strategies are put in place to make sure the deadlines are attainable.
“When people use terms like ‘aggressive schedule,’ that’s all fine, but it only works if you have a very clear strategy about how you’re going to achieve an aggressive schedule,” Westney said. “If, for example, when you say an aggressive schedule you mean the time from discovery of a reservoir to the production of first oil, and that’s going to be shorter than anyone has ever achieved before for a project of this size… you should have strategies that are designed to do that and not just say that we have an aggressive schedule.”
Briel et al. defined overconfidence as an underestimation of the variance of potential outcomes with regards to production. Companies are so confident in their projections that they believe any deviation from it will not be great; Fig. 2 shows those projections do not match historical experience. On average, project teams expect to deliver from 75% to 120% of their planned production, but the actual mean is only 81%. Three out of every four projects do not have an attainment of 100%, and the bottom quartile of projects only reached an attainment of 62%. The failure is not just in the immediate aftermath of first oil; even after years of production, the actual volumes of production fall short of those projected (Fig. 3).
So, does it stand to reason that this issue can be solved if companies just took a more cautious approach to evaluating projects? Cabano said it is a catch-22 situation: It would benefit a company to take more time, but given the demands of the industry, it cannot afford to take too much time.
“You want to be aggressive, but achievable,” he said. “You are in a very competitive marketplace right now… so, when companies are evaluating where to invest their money—and a lot of them have quite a bit of cash that they’re sitting on these days—they’ve got to spend it wisely, but spend it aggressively to compete in the marketplace.”
Conservative Response
If overconfidence leads to unrealistic projections, what happens when the cracks begin to show? Could companies do more to head off problems as they develop over the course of a project? The answer depends on whom one asks, but a closer look shows that E&P projects can offer little incentive for anyone to take initiative.
By their sheer size and scope, many E&P projects provide a safe cover for mistakes (Nandurdikar and Wallace 2011). Table 1 shows the survey results of project teams over a 6-year period. The teams were asked questions about their production forecasts at the point of sanction: were they given the same attention as their cost and schedule projections, were they peer-reviewed, and were they reviewed and approved by the entire team. The vast majority of those teams said their forecasts fit the three criteria. Asked to assess the quality of their production forecasts, three out of four teams rated them as good or excellent.
Nandurdikar and Wallace claimed that Table 1 is evidence that each function within an organization is responsible for a project not meeting its production estimates. This makes it easy for teams to blame other teams, so no single team or individual is accountable for a project’s failure. But while they say this is a problem for projects, Cabano said this is part of the reality. A project has many moving parts. Placing the sole blame on the manager, or any one team, is a difficult process.
“I think it’s more of a case-by-case deal,” he said. “If you have a project manager who didn’t follow the rules or didn’t document well, and he ends up with a bunch of claims and it doesn’t meet its expectations, do you hold him accountable? Yes, probably, but you don’t know all the details. If somebody lucks out and doesn’t follow the rules, but the project turns out great, right on schedule and on budget, do you give him all the accolades? Each project has a life of its own, so it’s hard to judge.”
Chaudhury and Whooley (2014) argued that the oil and gas industry operates at a statuesque pace when it comes to adopting new technologies and potential solutions to the problems that projects come across. Organizations have a natural reluctance to think outside of the proverbial box even as engineers develop innovative technical solutions, and without visionary executive leadership embracing these solutions, the industry will eventually reach its technical limits.
Knut Eriksen, senior vice president of business development at Oceaneering, agreed with the idea that owners tend to be slow to embrace unconventional strategies and technologies, saying the industry fosters this environment. Eriksen has experience working on deepwater projects for companies like Aker and Unocal.
“We are afraid that being too radical or thinking too much out of the box may not provide the predictability and quality of outcome that we need,” he said. “No innovation here. This project must be on schedule and on cost, and if you have a new product development in the project execution phase, you’re likely to lose control of this project. That’s been a mantra.”
What is at the heart of this conservatism? In an industry defined by its risk-seeking nature, what makes companies turn risk-averse once problems arise in the execution phase?
One place to look is at the costs. The money companies invest in projects leads to caution when they are faced with “out-of-the-box” strategies because no one can say with certainty that they will work. A failed strategy may lead to financial troubles.
It is hard to dispute that projects, facilities, and construction are costlier ventures now than in the past. The IHS CERA Upstream Capital Costs Index score, which measures the construction cost of new oil and gas facilities, was at 233 at the end of the second quarter of 2014. The IHS CERA Upstream Operating Costs Index score, which measures the cost to operate facilities, was 202. The values are indexed to the year 2000, so it costs approximately twice as much to build and operate a facility in 2014 as it did in 2000. Total upstream capital expenditures rose to USD 764 billion in 2013, an increase of 6% from the previous year (IHS 2013).
“If I’m going to invest a boatload of money here, I want to make sure you’re going to execute, so you need to prove to me that somebody’s been able to do that in the past and what measures they took to achieve it,” Cabano said. “Well, are you going to try something new in that kind of environment? It kind of handcuffs your ability to… try something brand new and, if it doesn’t work, learn from it and do better the next time. The project environment doesn’t do well in that scenario.”
There is also the fear of a potential incident on the scale of the US Gulf of Mexico Macondo blowout (2010). Eriksen said that it is easier for companies to work with proven technologies because it will better mitigate failure.
“In certain ways we just can’t afford to have disastrous failures,” Eriksen said.
The Influence Curve
Conventional thought within the industry is that owners assume more risk early in the project, during the definition phase. Once a project reaches sanction, the execution risks are low enough for owners to pass them off to contractors. This line of thought, also known as the influence curve, is the foundation of the front-end loading system that companies use to manage projects as well as the traditional best practices that companies rely on to address situations that come up during a project.
However, Westney questioned the effectiveness of the influence curve. An owner’s risk exposure—which he defines as a function of impact, probability, and ability to control an outcome—is actually at its lowest in the early stages of the project, when he has complete control, and increases dramatically after sanction, when the work is being done in the field and external forces have a greater chance of influencing a project’s progress (Fig. 4). The problem a company faces when focusing too much on front-end definition, he said, is that it creates a false sense of confidence. A company may lull itself into believing that all risks are mitigated prior to sanction and that effective front-end definition alone reduces execution risks to a point where they can be passed off to a contractor.
“It creates the idea that I don’t have to worry about anything else,” Westney said. “I don’t have to worry about all these risks that a megaproject can encounter. Once I start building it, now I’ve invested money into it, I can’t turn back and the impact of these risks hitting the project is much higher,” he said.
“This false sense of confidence closes people’s eyes to the idea that maybe there are some other things that can happen to your project that you ought to be paying attention to.”
Westney emphasized that he does not believe the influence curve is wrong, but rather that the way owners extend its meaning can be misleading, since they may not appreciate their post-sanction risk exposure. Cabano agreed with that assessment, saying that owners should accept that they own all the risk in a project, whether it is on the definition or execution side.
“We’ve placed a lot of emphasis on front-end loading to reduce risk exposure, but we need to be mindful of the fact that a project doesn’t go on autopilot after sanction,” he said. “We have to execute according to plan to assure we manage those risks effectively. Owners have to decide who’s better equipped to manage the risk, whether it’s themselves or the contractors or the suppliers or however you want to structure the execution portion of the job, but the owner owns the risk. Even in a lump-sum situation, if a contractor gets into trouble, it’s going to come back to the owner either through changes or claims or something later on.”
If one presumes that owners incur significant risk during the execution phase, the question then becomes: Is the work done after sanction more important to a project than the work done before sanction? Should the front-end loading system be a back-end loading system instead?
Perhaps not. It may be that the emphasis on the definition is not flawed. Instead, the emphasis may be inadequate, or the necessary work is not done to make front-end loading useful.
Cabano said that owners need to make the people who will be vital to execution an equally vital part of the front-end definition and get their input on what they are capable of producing. This belief has been expressed elsewhere (Frampton 2014), but there is a trade-off at play. Prior to sanction, the project does not exist. There may not be enough money to bring in a contractor, a construction manager, or a procurement team. Bringing in extra people too early in the process could have negative financial results if the project does not get sanctioned.
Cabano acknowledged the trade-off, saying that it is another difficulty that comes with project management.
“We’re not really sure if the owner company is going to approve anything yet, so I don’t want to bring this whole big team in,” he said. “At what point do I bring them in to where I’m not starting them too early, and if the project doesn’t get approved, I’ve got to lay them off… versus getting them in there to support the development of this whole front-end package and execution plan so that, when it does get sanctioned, they’re ready to roll?”
Fig. 5 shows the relationship between cost growth and the completeness of front-end work. As the quality of front-end work decreases, the cost overruns rise. Merrow (2012) concluded that the average E&P megaproject fell at the middle of the “fair” range of the front-end loading index, while non-E&P megaprojects were in the upper half of the “good” range.
“There will always be execution risk, of course, but the big issue here is all about understanding risks and then planning appropriately how to manage them,” Eriksen said. “The best tool for managing risk is knowledge. The best way of effectively influencing the outcome of a project is through early and thorough planning. There will still be some execution risk, but you can take out a lot of risk by understanding and performing a whole range of actions in the planning phase.”
Westney said he also believes that some execution risks can be mitigated by effective work in the definition phase, but not all of them. For example, if the demand for skilled labor and supervision was high at a time when the construction marketplace was busy, that would add significant risks to the construction costs and schedule that are not mitigated by front-end definition.
“Recognizing that owner risk exposure increases after sanction, it is logical to assume that owners need to be careful to structure the way they allocate risks and control in their contracting strategy,” he said. “The conventional owner perspective that ‘the more risk I can get a contractor to take, the better’ needs to be replaced with a more enlightened view that ‘the more I can optimize the allocation of risk to be consistent with my business goals, the more likely I am to succeed in meeting them,’” Westney said.
Consistency and an Eye for Detail
There are companies in the industry that place a premium on risk assessment, such as Technip, which has developed a risk management system (RMS) to handle large-scale projects.
Technip’s approach to risk and opportunity centers on four steps: identification, evaluation, response, and control. After analyzing a project, the company identifies and describes the potential events and consequences of any given action along with the potential causes. During this phase, project teams work with experienced people at meetings and risk assessment workshops. Table 2 shows the categories in which the company organizes the risks it encounters.
After identification of the risks, the company makes a qualitative assessment of the severity, the likelihood of occurrence, the level of criticality to the project, and the potential manageability. Response to significant risk involves taking the right action and developing the right plans to mitigate those risks. The company lists four goals that any response should have: the elimination, toleration, transfer, or reduction and control of risk. And then it reports and monitors these risks and mitigation plans until closure (Technip 2012-1). The process is the same when assessing opportunity, except the goals are different. A response will ignore, exploit, share, or maximize opportunity.
The RMS also accounts for uncertainties in cost and scheduling through predictive modeling using the Monte Carlo method to calculate likely results of a project and the Bayesian network model to help project teams manage uncertainty in decision making. Its contingency recommendations are based on a combination of the models and risk assessment.
Technip’s system has helped the company manage or comanage large-scale projects like the Gdansk Refinery for Grupa Lotos in Poland and Petrobras’ semisubmersible platforms 51, 52, and 56 in the Campos basin offshore Brazil. Noteworthy among its work are the eight spar platform hulls (Neptune, Nansen, Boomvang, Gunnison, Red Hawk, Constitution, Lucius, and Heidelberg) it has delivered or will deliver for Anadarko over the last two decades. That number represents almost half of Anadarko’s output and 40% of the spar hulls currently in operation or delivery worldwide (Technip 2012-2).
The Lucius spar was delivered in July 2013. It is scheduled to produce first oil in the fourth quarter of 2014 and expected to produce 80,000 BOPD. Heidelberg, a truss spar similar in design to Lucius, was more than 70% complete as of February and is scheduled to finish on time for first oil production in 2016 (Anadarko 2014).
Other Solutions
How can E&P projects run more efficiently? It is a question that the industry has been trying to answer for decades.
The question to be asked, perhaps, is whether the problems are really worth fixing. The answer may depend on the definition of failure. In discussing the lack of accountability in E&P projects, Nandurdikar and Wallace mention the problem of high oil prices; namely, that those prices allow projects to earn money even if they do not meet their production estimates, so a project could underperform and still be successful. From that perspective, a company may not see a project that left money on the table as a failure, even if it technically was.
Cabano pointed to another issue: In some cases, different members of a project team may have different definitions of success, and those definitions may not involve a project meeting its estimates. While an engineering team may be concerned with making sure its equipment and designs function properly at startup, with little thought to life cycle revenue, a business team may be concerned solely with revenue. In some cases, success may be as simple as keeping people employed for a certain amount of time.
“[A company] goes in with a certain scope of work, that project has a certain budget and a certain schedule, and I look at similar projects and how they’re executed for their cost and schedule, and that obviously they’re done safely and they operate well,” he said. “The business objectives may be very different. I don’t care what it costs; I just need to capture the market. Those business drivers don’t always align 100% with the project. You hope that they would, and they should in order to efficiently run a business, but a lot of times they’re not aligned.”
A common principle authors refer to when discussing optimism and overconfidence is the optimizer’s curse, or inevitable disappointment. Under this principle, when a decision maker consistently chooses alternatives with the highest estimated value, his or her selection will lead to a negative expected surprise, or disappointment, even if the estimated value is conditionally unbiased (Smith and Winkler 2006). The decision-making process exhibits a positive bias because of random errors in the estimation of project values.
McVay and Dossary (2012) applied this principle to oil and gas projects. They claim that inevitable disappointment is not inevitable because the random errors in estimation come from the biases of the estimator, which they define as a combination of overconfidence and directional bias, found in assessments with limited resources, like the front-end definition phase of a project. With no estimator bias, there is no expected disappointment. While the authors acknowledge that the elimination of estimator bias and random errors is unlikely, they state that with effective uncertainty quantification practices, expected disappointment and expected decision error can be reduced to negligible levels.
How does one limit overconfidence? By accounting for what McVay and Dossary called the “unknown unknowns.” They suggested that companies compare the performance of past projects to the projected outputs and then adjust the projections of future projects accordingly. It is a process based on an experiment designed by Capen (1976) and, while it sounds easy, the authors acknowledge how much it requires of companies in terms of discipline, institutional memory, and an appreciation for the lack of immediate results. In the current environment, that is a tough pill to swallow.
Westney said nothing short of a change in the culture of E&P projects is necessary. He outlined this culture change on two fronts, the first one being a company’s internal culture (how it views the people who create and manage the projects) and the second being an external culture.
“The oil and gas industry and the engineering and construction industry have a long tradition of tension between the companies that buy services and those that deliver them,” Westney said. “A lot of progress has been made, but there’s still a lot of progress to be made.”
For Further Reading
OTC-25344 Art, Science, and Engineering of Managing Offshore Field Development Economics and Risks
by G. Chaudhury and A. Whooley, Wood Group Kenny.
SPE 5579-PA The Difficulty of Assessing Uncertainty
by E.C. Capen, Atlantic Richfield.
SPE 145437 Failure to Produce: An Investigation of Deficiencies in Production Attainment
by N. Nandurdikar and L. Wallace, Independent Project Analysis.
SPE 160189 The Value of Assessing Uncertainty
by D.A. McVay and M.N. Dossary, Texas A&M University.
SPE 162878 The Economic Folly of Chasing Schedules in Oil Developments and the Unintended Consequences of Such Strategies by N. Nandurdikar and P.M. Kirkham, Independent Project Analysis.
Anadarko. 2014. Anadarko Announces 2013 Fourth-Quarter and Full-Year Results (3 February 2014).
http://www.anadarko.com/Investor/Pages/NewsReleases/NewsReleases.aspx?release-id=1896348 (accessed 10 September 2014).
Briel, E., Luan, P., and Westney, R. 2013. Built-in Bias Jeopardizes Project Success. Oil and Gas Fac 2 (2): 19–24.
Frampton, S. 2014. Major Capital Projects—Increasing Safety Assurance in Construction, Transport & Installation, Hookup & Commissioning. SPE Gulf Coast Section Projects, Facilities, and Construction study group webinar, http://eventcenter.commpartners.com/se/Meetings/Playback.aspx?meeting.id=756131 (accessed 21 August 2014).
IHS. 2013. IHS Upstream Spend Report: Global upstream oil and gas industry spending (historical and outlook), third quarter 2013 (11 November 2013). www.ihs.com/products/cera/energy-report.aspx?id=1065984721 (accessed 5 September 2014).
Merrow, E. 2012. Oil and Gas Industry Megaprojects: Our Recent Track Record. Oil and Gas Fac 1 (2): 38–42.
Smith, J.E. and Winkler, R.L. 2006. The Optimizer’s Curse: Skepticism and Postdecision Surprise in Decision Analysis. Management Science 52 (3): 311-322.
Technip. 2012-1. Risk management brochure. http://www.technip.com/sites/default/files/technip/publications/attachments/RiskMgmt.pdf (accessed 11 September 2014).
Technip. 2012-2. Technip to start construction work for the Heidelberg Spar (27 November 2012). http://www.technip.com/en/press/technip-start-construction-work-heidelberg-spar (accessed 11 September 2014).