Big Data and the Next Big Thing

Getty Images

SPE presidents are asked a lot of questions, and I suppose that’s true of anyone who stands in front of an audience as an authority figure. I have learned to expect many of the same questions, no matter the audience: What will the price of oil do? When will the industry turn around? How is the industry managing post-Big Crew Change? What are you going to do to “fix” the situation of women in the industry? What is your view on climate change?

But my favorite question is: What’s the Next Big Thing? As I travel around as SPE President, I see a lot of new and different technologies, operators, and ideas. But my view on the Next Big Thing has been consistent for the last few years: I believe that we are not only entering the era of universal big data, but also the era of more intelligent software to help us make better decisions. The software will help us be better engineers by preserving knowledge and improving decision making.

I believe the Next Big Thing will be infinite big data and intelligent machines to interpret it. Cheaper and more robust sensors will allow us to collect data not only on expensive deep­water completions and big compressors, but also on every isolated onshore well. Cheaper computing power, universal communications, and learning algorithms will allow engineers to use and interpret those large amounts of data to manage assets more efficiently. I believe operators and service providers will be able to reduce risk by using the full value of knowledge and experience to optimize asset management.

I learned to be an engineer in the analog world of the late 1970s and early 1980s. Ours was a world of hand-drawn maps and handwritten columns of figures on green engineering paper. Programmable handheld calculators replaced the slide rule. Data capture was infrequent, monthly at best, and analysis was slow and tedious.

Enter the PC. I vividly remember first getting a PC at work in 1985 after I made the pitch to our district engineer to purchase an IBM PC Model XT for our six-person team. Before then, only special programs could be run on the mainframe at the faraway research center. Our group managed several Permian Basin waterfloods, and we had been doing very tedious calculations by hand. When I was pitching to the district engineer, his first question to me was “how many engineers will this PC replace?” What he didn’t see was that computational power would help the same number of engineers create better analysis and help drive better decisions. Since those ancient days of more than 30 years ago, better, faster, cheaper computing power is everywhere. We haven’t replaced engineers with PCs, but we have created new opportunities for better technical and economic evaluations.

Enter the Internet. When I left ARCO in June 1994, engineers did not have email or Internet access. Three years later, I was emailing work across the Atlantic every day so that teams in Houston and London could collaborate. Now, especially for those of us in global companies, our workflows are completely dependent on instant global communications. But I believe we are only beginning to take advantage of the full potential of the globally connected computing world. The Internet, or Cloud, of Things is coming for all of us.

Another question I’m often asked when I speak is: How will we manage the Big Crew Change? What will the industry do when all these people retire? How are we retaining the knowledge of all those experienced practitioners? I believe that many of my peers who are retiring or have retired as part of the Big Crew Change will continue to consult or work part-time on their own terms. But I think the greatest value will be for these experienced practitioners to help teach not just the next generation of human staff, but also the next generation of intelligent software. Their knowledge can be preserved forever in software that has learned how to duplicate, and eventually improve, their decisions.

In his book, The Inevitable, Kevin Kelly refers to this artificial intelligence as “cognification”—not making a computer think like a human, but inserting a degree of “smartness” into its analysis. We already see the beginning of learning software with Google’s search algorithms and Apple’s Siri’s knowledge of everything. Intelligent software that brings efficiencies to humans is everywhere. Modern commercial aircraft are flown mostly by software. Routine radiology interpretation is being done more often by software, with physicians checking the work. Self-­driving cars learn your habits and are improving rapidly. There is no reason not to expect that the oil industry can use artificial intelligence and learning software to preserve and replace the human experience that is leaving the industry.

At an SPE meeting not long ago, I overheard a familiar voice at a nearby table. It belonged to a Permian Basin fracturing expert who designed fractures for me in Midland, Texas, in 1981. He has forgotten more about hydraulic fracturing in the Permian Basin than most will ever know, and he is still designing fractures and mentoring younger technical staff. He is a perfect example of deep technical knowledge that could be preserved through learning software. Could you ever replace what he knows? Not completely, but software can help preserve it.

Better sensors and intelligent software can help engineers manage complex operational systems. Our more complex production systems are too difficult for an individual to understand all the interactions. In the 1980s, a driller manually worked the brake, watching the gauges and pits. Today, the iron roughneck is a robot that has replaced rig hands while the rig operator works the controls from the safety of a booth. In a 1980s fracturing job, the company engineer (me) sat in the control van with the service company engineer monitoring pressures and rates as we manually adjusted the pumps. Today, fracturing controls are computerized and can be watched from the office.

Innovation isn’t always a completely new invention. It is often the merger of two old techniques with a new idea. A great example is hydraulic fracturing+horizontal drilling=the shale revolution. Cheap, robust sensors+cheap computing power+learning software=truly real-time asset optimization. And the more that learning software is used, the more it learns and the better it gets, just like Siri. But software won’t replace human judgment; rather it will help us make better decisions with better access to data and better analysis of the data. For example, software can be used to improve subsurface interpretation by sifting through big data, assisting the human interpreter. To learn more, check out this article in the September JPT, “Better, Less Tedious Fault Interpretation” (

I like to think of big data plus cognification in a Star Trek way—like Captain Kirk and Mr. Spock. Spock was all logic and data, while Kirk was all emotion. Working together, the two could solve most any problem. Access to our own personal Spock of infinite logic and knowledge will certainly improve ­decision making. Software cannot replace the innovation that relationships and personal interaction brings—those “aha” moments at a meeting or in casual conversation with a colleague. Big data will make us better engineers, but relationships—through SPE—spur innovation.

At the 1964 New York World’s Fair. (I’m on the left.)


As a child, I visited the 1964 New York World’s Fair. IBM displayed the wonders of modern mainframe computing, and the Bell System’s exhibit included the picture phone and modem, both futuristic innovations in an era of rotary dialing. We could not have imagined the innovations of 2016. In 1986, we could not imagine the computer revolution, the Internet, and apps.

Similarly, the revolutionizing inventions of 2046 are beyond our imagination. In my view, big data and learning software are the Next Big Thing, and they will revolutionize upstream, midstream, and downstream operations.

Big Data and the Next Big Thing

Janeen Judah, 2017 SPE President

01 November 2016

Volume: 68 | Issue: 11

No editorial available



Don't miss out on the latest technology delivered to your email weekly.  Sign up for the JPT newsletter.  If you are not logged in, you will receive a confirmation email that you will need to click on to confirm you want to receive the newsletter.