Big Data analytics and applications, the Internet of Things and other technologies are designed to make operations more efficient, secure and safe. Operators and service companies already are seeing the benefits of these technologies, but with any project challenges can arise, and game plans need to be put in place.

For this roundtable, E&P interviewed Ole Eyvind Evensen, IBM upstream strategy leader; Earl J. Dodd, supercomputing strategist and architect with Ideas and Machines Inc.; Randall Hoppe, industry director for oil and gas at Microsoft; and Lisa Davis, a member of the managing board of Siemens AG.

E&P: What are some of the major challenges that exist for oil and gas companies in regard to Big Data analytics and applications? In which areas are your clients asking for the most assistance?

Evensen: Oil and gas operations cover several very different but interdependent disciplines.

The challenges they face include capturing, safeguarding and sharing more volumes of data that can be tapped for new insight.

The foundational data management challenge is being addressed in a situation where there is a shift to the cloud, adding an infrastructure element to the equation.

The opportunities we are discussing with our clients are to shift data to the cloud, which allows for new services while being able to explore a new generation of Big Data analytics that can be utilized across regions.

Price, security and flexibility are key considerations.

Well delivery (drilling) is facing a major change with the need to improve real-time decision-making capabilities. The velocity and volume of drilling data available are posing a challenge for the disciplines involved. Compared to exploration they face a wider challenge to be able to access and use their historical data. These data are a vital source for insight, to develop predictive capabilities that can be applied during operations and to avoid nonproductive time and undesired incidents.

A highly instrumented well may generate more than 2 TB of data per day during drilling, which requires skills and unifying processes as well as suitable technology to manage streaming Big Data. The typical client request in this area is to help develop predictive capabilities, which can be implemented into existing operating models.

Operations are facing a similar trend of increasingly digitalized facilities. An offshore rig may have more than 70,000 sensors. They are potential sources for insight to understand equipment condition and overall system performance. Looking to other leading industries such as automotive and aviation allows a view of the art of the possible. Our clients are asking us to share lessons learned from other industries and to help them create a digitalization roadmap for safer, leaner and more efficient production.

Dodd: The top technology challenges for today’s oil and gas companies around Big Data analytics and applications concentrate around algorithm scaling, algorithm robustness, data saliency, workflows security, and skills development and retention.

With the growth and demand of (near) real-time processing of data, the oil and gas industry should investigate and move toward functional programming languages and tools. The industry should come together to extend the current functional language base, form a working standard and develop application- kernels migration tools based on this functional language suite. Only functional languages and tools can address the future of algorithm scaling and robustness.

Data saliency at real-time speed is a problem that is not receiving enough attention from academia and industry. ‘Garbage in is still garbage out,’ as I learned over three decades ago. What’s missing are the skills to understand our complex workflows and what represents data saliency and output appropriateness. Skill assessment, retraining and retention go hand-in-hand with the Big Data challenges.

An immediate concern for the oil and gas industry is the security vulnerability of our workflows and their parameterizations. Data per se is not at risk; your workflows are in danger. If I can leave the reader with a single call to action, it would be this: Secure the workflow setups and the parameters used.

Hoppe: In 2016 Microsoft and Accenture commissioned an oil and gas digital trends survey that identified five key trends in the oil and gas industry that focus on data and analytics.

The first trend is that digital technologies like Big Data and the Internet of Things are recognized as adding value to upstream oil and gas companies by helping to reduce costs, speed decision-making and increase work productivity.

Secondly, oil and gas companies will continue to invest at the same amount or slightly more in digital technologies despite low oil prices.

Third, digital investments in oil and gas are focused on enabling the mobile worker and using the Internet of Things for predictive analytics. This trend is expected to continue for the foreseeable future.

The fourth trend is that gaining maturity in analytics will be key to reaping the benefits of Big Data and other digital technologies. Finally, the cloud is becoming increasingly important to oil and gas companies as a stepping stone for other digital technologies.

As upstream oil and gas companies prioritize investments in their organizational capabilities, they’re spending smarter today on digital technologies.

Davis: Longer term volatility in oil prices requires a step change in the oil and gas industry to permanently lower production costs.

That means exploring different ways of thinking and doing business to remain competitive.

While the oil and gas industry is no stranger to Big Data, properly harnessing those data to their fullest potential is a powerful antidote to volatility and rising development costs.

In oil and gas digitalization brings a convergence of IT and operational technology connectivity that enables data to travel from the field to the control room to the enterprise network, underscoring the need for a unique set of solutions to address the crossover between IT and operational technology.

Few owner-operators are ready to take the full ‘digital plunge’ due to concerns such as cybersecurity, costs and data confidentiality. We work closely with customers to explore and implement digital solutions that can be scaled up incrementally and ultimately integrated into a full-fledged digital enterprise framework. This scalable approach minimizes risk and interruptions, enables the investment to be spread out over time and enhances the ‘comfort level’ at all levels of the organization.

E&P: What strategies can oil and gas companies employ to improve their uptake?

Evensen: Leading oil and gas companies may choose to be a first mover or a fast follower. There is no real third option. Leaders are advocates of improvement and change and take pride in strategic initiatives. They have a clear vision of where they want to take their organizations and recognize the importance of people buy-in.

A good approach to secure buy-in is to include all affected disciplines in both the target setting and a critical review of the processes being affected. New data allow new insight and shared situational awareness that can influence how people can and should work and collaborate. Without a holistic approach, including the organization, the processes, technology and old as well as the new Big Data will expose the effort to weak links.

Dodd: Strategies for improvement are based on long-term return value to oil and gas companies and include the following areas, leveraging a top-down approach: Rethink the organizational structure for business process performance and optimization; rethink the workflows for digital transformation and cybersecurity; and rethink the entire data life-cycle model and its value chain across all stakeholders.

The industry is not in a data sprint; we are in a workflow optimization marathon. Agility and suitability are the new capabilities of the digitally transformed organization. A digitally transformed oil and gas company has core capabilities supported by organizational enablers. Data and algorithms are part of one’s core capabilities and require sustainable enablement.



Services-driven (including all touchpoints)

Strategy, planning and governance

Workflows (interaction and automation)

Operating model and organizational structure

Decision support (analytics with scale and scope)

Leadership and culture

Data (saliency, access and stakeholder sharing)

Technology and technique (agility and suitability)

Resilience and performance

Talent (skills development and retention)

(Data courtesy of Earl Dodd, Ideas and Machines Inc.)

Hoppe: Caglayan Arkan, Microsoft’s general manager of worldwide manufacturing, recently commented on how businesses can stay relevant and competitive in today’s digital era in the blog Digital Transformation: Seven Steps to Success.

One of the points he made was that leadership matters. Leaders need to understand how technology will impact their business, and they will need to develop their vision and communicate it.

Organizations should drive culture change through effective change management. Organizations need to communicate what the new mindset should be, what the new value propositions are and how employees are expected to perform.

Businesses also need to connect their customers, products, assets and people. There are more data to collect, understand and get insights from, which will inform an organization’s next steps and how to take action.

Businesses adopt a data culture. Organizations must shift from making decisions based on habit, opinions or experiences to making decisions based on data. It is important to be data-driven and understand what’s in the data.

And organizations should experiment and fail fast. Today’s digital era is about experimenting in monthly if not weekly cycles. Businesses need to find the use case, get the data, formulate insights and act. If it fails, go to the next use case.

Davis: Everybody is talking about digitalization in a different way. At Siemens our focus is on converting data into value. We believe that efficient product flow begins with efficient data flow. Automated equipment produces a constant stream of data—measurement data that can be mined, aggregated into Big Data and transformed into smart data through intelligent analysis. Smart data helps us understand production processes better. The more data the system generates, the more possibilities we have to influence and improve individual processes.

E&P: Why would you urge operators in the oil and gas industry to turn more of their focus on Big Data analytics and applications?

Evensen: Organizations must acquire and demonstrate new capabilities to operate differently in the years ahead. ‘Doing better with less’ implies fewer human resources, capex, opex, waste and tolerance of environmental risk. New technology is the primary enabler of this change. It will require mastery of legacy data and corporate knowledge as well as new data sources to obtain new insight for better and timelier decision-making. The reward of this effort and investment is reduced production losses, improved efficiency and automation as well as safer operations and a platform for organizational learning.

Dodd: We are moving beyond digitization into the real-time digital transformation of secure operational excellence. The industry continues to pass through the era of digitization, yet it has not made the full leap into the digital transformation realm. Big Data, analytics, applications and algorithmic approaches are capability tools supporting digital transformation, but these tools are not enough. Organizations need chief digital transformation officer alignment within the C-suite to keep the cadence for business performance optimization and transformation. Time is of the essence for digital invention and innovation at real-time speeds to build an agile and suitable operating model.

Big Data and advanced analytics are rapidly becoming a key determinant of competition across all industries. Yet the projected demand for deep analytical skills leveraging Big Data will exceed the supply of talent. This talent is difficult to produce (e.g., specially trained with industry knowledge). This is a constraint for growth and sustainability. The oil and gas industry must begin now to develop, retrain and retain skilled workers to address the talent gap. One cannot fill this gap by changing university requirements and wait for graduates to make it through the educational pipeline. It is necessary to retrain the talent in place today.

Hoppe: Recent advances in digital technologies have greatly enhanced how organizations can leverage digital technologies for their competitive advantage. Cloud computing services that facilitate the Internet of Things, analytics and connected field service allow organizations to focus on business priorities vs. technical architectures that support them. For example, high-performance computing enables oil and gas companies to run complex models in record time, reducing the need for costly capital investments in compute infrastructure while delivering results in a fraction of the time. Organizations that understand and leverage technologies available to them today will enjoy a lower cost of operation, safer operations and a more efficient supply chain.

E&P: What do you think the technology trends of the future will look like?

Evensen: For the oil and gas industry the trends can be identified by looking at faster moving industries such as automotive, aviation, healthcare and financial services. They are already adopting technology the oil and gas industry is assessing. Among the key trends we expect to observe are increased automation in several operational as well as administrative areas. Operational automation will rely on better access to, and analytics of, increasing volumes of sensory/Internet of Things ‘data in motion.’ Administrative ‘robotics process automation’ will impact back-office functions and be a natural part of business process improvement initiatives. Equally important is our need to better understand and make sense of internal and external unstructured data. About 80% of an organization’s data are unstructured, which cannot be analyzed with traditional analytics technology. Cognitive analytics will enter all disciplines and create opportunities for competitive advantages. Geoscientists will utilize cognitive technology to better understand prospects and even mature fields by identifying relevant analogs from thousands of studies, reports and papers. Understanding technical documents with pictures and illustrations as if they were ‘structured data’ will provide an information advantage and soon be considered a foundational capability.

Shifting the focus from IT operations to IT enablement will most likely accelerate outsourcing and adoption of cloud platforms and services. As cloud platforms and services are appearing open and interchangeable, hybrid clouds will emerge as a preferred model to accelerate, simplify and finance faster adoption of technology that enables new operating models.

Dodd: My high-performance computing, Big Data and analytics predictions for the oil and gas industry are: • 2018: Blockchaining of workflows, allowing secure interoperability;

  • 2019: Cloud computing in hybrid bursting deployment implementations;
  • 2020: Specialty accelerators for distinct classes of workflows;
  • 2021: Industry-led functional language and toolware standardization;
  • 2023: Real-time data commoditization and democratization; and
  • 2025: Quantum computing for optimization problems.

Intellectual property contained in the parameterization and sequence of our workflows must be secured. Cybersecurity methods like blockchaining will become essential for workflow orchestration.

Hybrid cloud computing will eventually be commonplace and accepted by oil and gas operators. Cloud bursting requires cybersecurity of the workflows, though.

Specialized processing and proprietary systems become common again. Oil and gas will see modular systems that are specialized for particular aspects of the workflow. Field-programmable gate array and dataflow computers will return. Cognitive computing will address pre- and post-processing automation.

Scalability and robustness of algorithms will need a rethink on how to reprogram applications and libraries. Functional languages will make a dramatic resurgence for scientific and technical computing.

Computing is a commodity. The cloud has democratized computing and data. Real-time Big Data for processing data-in-motion problems will become commoditized and democratized.

Quantum computing will become a reality and will revolutionize continuous global optimization problems like reservoir simulation and production scheduling.

Hoppe: While it is challenging to predict in what state technology will evolve, there are technology trends that give us hints as to what it may be like in the future.

While the concepts of the Internet of Things [IoT] has been with us for decades (collect data, analyze data, act on said data), today’s IoT has allowed us to mash disparate data sources and run compute intensive processes (high-performance computing, predictive analytics algorithms, etc.) on those data. There is a wealth of data that is still not tapped (and will be) as technology advances and price points decline. Further insights will be gleamed from access to new datasets.

We will continue the journey from diagnostics to predictive to prescriptive analytics. Most organizations have conquered diagnostics and are dabbling in predictive [analytics]. As technologies to more easily model and build analytic algorithms progress, both predictive and prescriptive analytics will be applied to more and smaller problems.

[In regard to visualization] most organizations have made the leap from paper to glass. Visualization tools will continue to make it easier for organizations to bring data and analytical results together for newly found insights. And as visuals have moved from paper to glass, they will also move to 3-D holographic representations to more realistically represent data and analytics.

As the IoT, analytics and visualization technologies move forward, it enables operators, suppliers and service providers to offer ‘everything as a service.’ Imagine purchasing ‘flow’ vs. a pump. The operator transfers variable costs of operation and maintenance to a predictable fixed model, given appropriate service level agreements. Everything as a service applies to many types of equipment and business processes/services and will change the traditional supply chain ecosystem of all involved.