Decision making in uncertain environments is key to the successful delivery of oil and gas projects. By definition, however, uncertainty is ambiguous and unpredictable.
Uncertainty does not necessarily imply risk; the two are separate concepts. Economist Frank Knight made a clear distinction between the two as far back as 1921. Risk can be measured and quantified and is well suited to the analytical techniques used in project risk management; uncertainty defies quantitative expression. Items with the quality of uncertainty often surface during risk-assessment exercises but tend to be categorized as “issues” and are neither captured nor addressed by traditional risk-management processes.
There are known knowns; there are things we know we know. We also know that there are known unknowns; that is to say, we know there are some things we do not know. But there are also unknown unknowns; the ones we don't know we don't know. — U.S. Secretary of Defense Donald Rumsfeld, 2003
Donald Rumsfeld’s infamous quote highlights the difficulty of articulating uncertainty. However, the statement is rooted in sound cognitive and sociological theory and makes an important point about the perils of taking our individual and shared perceptions for reality. The philosopher Slavoj Žižek (2006) suggests that there may also be a fourth category of “unknown knowns”—those things that we intentionally refuse to acknowledge due to, for example, the social stigma around dissent. While we may feel confident dealing with “known knowns” and “known unknowns,” how can we hope to discover the “unknown unknowns” and uncover the “unknown knowns”?
These problems can be effectively addressed using a Delphi-based workshop methodology. The process presented in this article discusses how to work collaboratively to develop a common understanding of a project and achieve alignment on issues and uncertainties. This allows managers to gain a better understanding of the critical uncertainties running through the project and implement appropriate strategies.
Delphi Methods
Delphi is a structured communication technique that was developed as part of the United States’ response to global uncertainty during the Cold War. Since then, the process has been used across many different industries worldwide (Linstone and Turoff 2002). Delphi-based methods are able to draw out the tacit knowledge held by project members and assess it in a structured manner.
Delphi methods allow a diverse group of individuals to work as a whole when dealing with complex problems and include these features:
- Individual contributions of information and knowledge
- Assessment of the group judgment or view
- Opportunity for individuals to revise views
- A degree of anonymity for the individual response
Delphi methods are particularly beneficial when one or more of the following apply:
- The problem does not lend itself to precise analytical techniques (e.g., due to complexity) but can benefit from subjective judgments.
- The individuals who need to contribute to the examination of a broad or complex problem do not have a shared history; may not have had time to establish adequate communication; and may represent diverse backgrounds with respect to culture, experience, or expertise.
- More individuals are needed than can effectively interact in an unstructured face-to-face exchange.
- Long-term group co-location is infeasible.
- Disagreements among individuals are potentially severe or politically fraught requiring the communication process to be refereed and/or anonymous.
- The heterogeneity of the participants must be preserved to assure validity of the results, i.e., avoidance of domination by majority opinion or by strength of personality.
The authors have adapted Delphi-based approaches to complement and enhance typical engineering and oil and gas management processes. A Delphi workshop takes the form of a facilitated, guideword-led meeting. The workshop typically takes 1–3 hours, does not require any preparation work from participants, and the full results can be made available within 24 hours. It is appropriate for groups from 3 to up to 20 participants.
The purpose of the workshop is to identify, understand, and clearly articulate important project uncertainties so that appropriate management strategies can be put in place. The outcome is an integrated overview of the project, a summary of its key risks and issues, and alignment across all disciplines. The basic workshop procedure is described in Fig. 1.
A comprehensive keyword list of more than 300 prompts has been developed. Within the groupings, the keywords are arranged alphabetically. This dissociates a keyword from the preceding one and allows each keyword to create its unique discussion space. The list is designed to cover all areas, and can be adapted, shortened, or lengthened to best fit the project at hand. Participants receive no further guidance as to the meaning of a keyword and are free to make their own associations.
Participants are issued scorecards with numbers from the Fibonacci sequence (1, 2, 3, 5, 8, 13, etc.) As each keyword is read out, participants “play” their cards simultaneously based on their view of the magnitude of issues associated with that keyword. No guidance is given on which criteria constitute a high value vs. a low value. Participants are free to determine their own internal view of what the numbers mean.
If all participants award identical or similar low scores, the group may agree to quickly move on to the next keyword. If there is a wider spread of scores, or an outlier score, then the participant with the highest score begins a discussion on that subject. The discussion is continued until a consensus is reached. Participants are not required to score items outside their area of expertise.
The choice of a nonlinear scoring sequence reflects the increasing uncertainty associated with larger scores (Cohn 2006). It forces participants to rank issues in accordance with the difference in scoring magnitudes, while freeing them from getting hung up on assigning an exact number to each one.
Notes taken of the discussion are displayed on a large screen during the meeting as a means of focusing the discussion and verifying agreement. This ensures issues are articulated fully, clearly, and to the satisfaction of all participants. This also has the added benefit of integrating the first review cycle into the meeting. Participants are able to take the raw workshop output with them at the end of the meeting ahead of the issue of the formal report, which is typically delivered one day after the workshop. Only the discussion relating to the keyword is recorded. The scores assigned by the participants are deliberately not recorded to avoid the potential for an anchoring cognitive bias developing for either the participants or for onward readers of the report.
Benefits of the Approach
To date, the authors have applied this process on several projects. The method provides the following benefits in comparison to traditional risk-management methods:
Knowledge sharing. The process was able to take advantage of private information held by each of the participants. Participants’ diverse knowledge, experience, opinions, and eccentric perspectives allowed the collaborative discovery and articulation of areas of concern not being consciously addressed by the project as a whole.
Challenging held assumptions. The introduction of additional perspectives to project issues led to the downgrading of certain interesting but ultimately trivial issues that had until then been receiving significant attention. This was largely through the introduction of mitigating information provided by participants who had not been involved in the initial prioritization and subsequent management of these issues. Conversely, the introduction of these new perspectives also led to the escalation of a number of issues that had previously been seen as of low criticality or low priority.
Sense making. Open-ended and unguided input was successful in uncovering and aggregating information, enabling the discovery or crystallization of issues only partly understood by project participants. The process gave participants freedom to express concern with respect to “nagging doubts” and “gut feelings.” The flexibility of the process allowed subjective issues/risks to be expressed in a rich, contextualized form. This led to the emergence of patterns and a deeper understanding of project issues and their potential mitigations.
Time effectiveness. The meeting was fast and easy to organize. A typical workshop lasts 2 to 3 hours and international participants are able to arrive on a morning flight and depart in the evening.
Collaborative fixation/participant in-dependence. The tendency to provide input that conforms to ideas suggested by other participants was largely eliminated. This led to an increase in the quality and quantity of issues raised during the meeting.
Attention and personality management. The short duration and the planned pace kept the participants engaged throughout the workshop. The scorecards created independence in participants’ engagement, and the keywords introduced an element of gamification. Nonverbal scoring helped in managing the effect of dominant personalities and ensured everyone’s views were integrated into the consensus.
Consensus building and alignment. The process drove consensus within the group. This set the basis for future collaborative efforts in resolving the issues raised.
Conclusions
The Delphi-based approach allows a group of individuals with varying perspectives and expertise to develop a robust understanding of a complex problem. The collaborative process addresses many of the shortcomings of traditional risk-management approaches applied to industry projects and allows discovery of complex project risks and uncertainties that might otherwise remain unexpressed.
This approach provides management with an integrated and in-depth project understanding that enables them to react early and confidently to unfolding situations. The method provides an efficient means of running a workshop, has the ability to reinvigorate existing issue- and risk-management frameworks, and ensures project resources are focused on the most relevant risks and issues.
References
BBC. 2003. Rum Remark Wins Rumsfeld an Award. (accessed 25 June 2015).
Cohn, M. 2006. Agile Estimating and Planning. Upper Saddle River, New Jersey, USA: Pearson Education.
Knight, F. 1921. Risk, Uncertainty, and Profit, Boston, Massachusetts, USA: Houghton Mifflin Co.
Linstone, H.A. and Turoff, M. (Eds.) 2002. The Delphi Method: Techniques and Applications. Newark, New Jersey, USA: New Jersey Institute of Technology. (accessed 19 June 2015).
Žižek, S. 2006. Philosophy, the “Unknown Knowns,” and the Public Use of Reason. Topoi, 25(1): 137–142. (accessed 25 June 2015).
Lakshan Saldin is a founder and director of Agilis KLM, a consultancy specializing in helping clients execute projects and programs in uncertain environments. He has 20 years of experience in engineering and oil and gas projects. He has spent the last 10 years in a variety of senior operational and project management positions. Saldin is a chartered engineer and a fellow of the Institution of Chemical Engineers, and holds degrees in chemical engineering and project management.
Matthew Healey is a founder and director of Agilis KLM and has 20 years of experience in oil and gas and large infrastructure projects. He has worked in several management and advisory roles in Australia and Western Europe, with a focus on commercial and technical risk.
Kate Parker, SPE, is a founder and director of Agilis KLM and has 20 years of experience across a wide range of oil and gas projects including technical, management, and consultancy roles on a number of projects and industry firsts. She is a chartered engineer and a member of the Institution of Chemical Engineers.