Big Data in Oil and Gas Industry: The Journey

Big Data in oil and gas industry is no more a headline; it is the future in making.

We are humans and we are curious! Aren’t we?

Since time immemorial, it is the inquisitiveness of mankind that has enabled him to venture into an unexplored landscape and take chances. One such magnificent endeavor has been to move towards ‘digital revolution’.

Let’s travel 50 years back in time!

The Oil and Gas Industry stood tall as the pioneer of employing Distributed Control System (DCS) to regulate refineries and control the downstream plants. Being an early adopter of data processing technology, it is involved with large chunks of data i.e. the time series information which is recorded from the instruments that control and monitor the plant process. Though indexed by time and excess in volume, the data is well structured.

Then arrives Big Data in the picture!

Big Data consists of data in both structured and unstructured form. Here comes the role of Big Data Analysis which deals with drawing relevant insights from the data.

Smart Technologies like Cloud Computing, IoT devices, and other intelligent software plays a major role in meeting the demands consequently helping the operators. However, in spite of the advancement in technology, only 0.5% of the generated data is analyzed and used.

What is Big Data? How big is it?

To put it simply, it’s a term that describes a big amount of data of high velocity, variable and complex data that require sophisticated techniques to enable the storage, management, distribution, and analysis of the information to provide enhanced insight, process automation and decision making.

Yes, it is really big! But the “big” in Big Data doesn’t imply big in terms of volume alone but also the complexity, and the variety of sources from where it generates. So if an organization or industry faces any hurdle in dealing with the data velocity, volume and variety, then that’s the big data challenge. These challenges introduce the necessity for data management and analysis techniques.

3 Vs in Big Data

Volume: With sizes ranging from tera to zettabyte, a huge volume of data is involved.

Velocity: It refers to the speed at which the data is analyzed and the frequency at which data is generated.

Variety: The various sources and forms of data which are obtained from both internal and external sources usually range in different formats such as log data and transactions from a variety of applications; structured data; semi-structured data like XML data, video/audio statement, unstructured data like text images and more.

Purpose of Big Data in this industry

The technology advancement has made this industry safer, cheaper and cleaner, paving the way for huge investments and opportunities for the oil giants. In an oilfield, there are wide subdivisions. The emerging concept of Digital Oil Field demands for analysis and acquisition of a huge amount of data which are, generated during the various upstream processes, classified into wide subdivisions. These data are then analyzed to form the statistics and track the proceedings.

The major areas that analytics are focusing upon are:

• Geological framework: The accurate and precise attributes of the shale basins can be evaluated by juxtaposing the surface geology to the well performance data.
• Drilling: With better analytics, the number of wells can be minimized, so will the lag time.
• Optimization: The collection and analysis of this hybrid dataset improves a company’s capacity to optimize the drilling parameters.

Quality Data

Big data plays a very important role in our mundane saga. From driving directions to specified locations on our smartphones, it is everywhere!

Similarly, it plays a key role in pipeline operations. Let’s learn how.

• Data on possible instrumentation variable which includes transmitters, meters and analytical tools like gas chromatographs; Operating variables which include pumps, temperatures, flow rate and drivetrain.
• Historical Data includes:

1. Short term data which are collected at regular intervals for various incident investigations.
2. Long-term data collection which includes analysis and identification of trends at less frequent intervals
3. Event data like the human intervention on the equipment for preventive maintenance and replacements, equipment failure and many more.

Regardless of the advancements, a large proportion of pipelines are still not available for pig inspections.

Then and Now: How far have we come?

In 1992, the data generation per day was estimated to be around 100 gigabytes. Today, it’s going to reach to an estimated 50 terabytes per second!

Earlier, finding the right value in big data was cumbersome and lengthy due to its unstructured form. However, the real problem is not just about obtaining the right data, but to analyze and visualize the appropriate data from the cluster of data. For example, corrosion poses a serious threat to all type of infrastructure and creates disruption in various industrial operations.

Corrosion or cracks are the reasons behind pipeline failure. Pipelines, which are the most efficient way to transport crude oil and natural gas across thousands of kilometers, fail to serve its purpose due to corrosion. Preventive measures like inspection, evaluation and retaining of the asset by integrity management play a key role in dealing with such failures.

In-line inspection is a component of such management. Here comes the use of Smart pigs, which are mainly used to conduct such inspections. This device is inserted into offshore and onshore pipelines to execute various types of operations without hindering the fluid flow. Basically, there are two different types of technologies used which assess the pipe wall condition, magnetic and ultrasound. When inserting pigs, irrespective of the technology, a large amount of data is generated of varying sizes and formats which are then inspected and analyzed by the inspection official. The output provides the estimates of sizes of cracks or corrosion area with a varying level of accuracy which is considered for decision making.

Advanced analytics help to uncover the underlying patterns which are not known by conventional analytics, thus assisting in providing more valid predictions. Such forms of predictive analysis are the future of pipeline monitoring and assessment provides the stakeholders with an overall view of operations, thus resulting in managing the assets in an effective and controlled manner.

Prescriptive Analytics, on the other hand, are used in both unconventional and traditional wells which help in analyzing the future by providing information on what will happen, why, when and how to improve it. With the help of Smart pigs, the cracks and corrosion development in pipelines could be possibly predicted. Thus by analyzing the videos, data which are obtained from the cameras installed in the smart pigs, preemptive actions could be taken.

Thus, smarter decisions equal lower environmental impact, fewer resources, and higher yields. Successful organizations are the ones who know how to produce, transport oil and gas and prioritize resources to extract in the most effective and efficient manner.

Prescriptive analysis and Big Data can emerge as a big player in the coming years.

Let’s be on the lookout!

Way forward

Thus a fully automated and digitalized service provides enormous benefits to the standard operations, enhancing the efficiency of risk management programs. These analytical tools can help operators optimize maintenance regimens and enhance inspection intervals by intermixing industry, asset specific, real-time and historical data into a decision and predictive process.

Thus identifying anomalies and precisely predicting future performance will enable better decision making and reduce risk. Such digitization initiatives would help the Oil and gas industry make huge profits and provide opportunities in almost every facet of risk and asset management.

“Prospecting for oil is a dynamic art… The greatest single element in all prospecting, past, present and future, is the man willing to take a chance.”

                                                                                                              -Everett DeGolye

With the right intent and efforts, the chances are high that we will succeed!