Latest Posts

Featured Posts

Filter By Categories:

An introduction to the Big Data Landscape

By Roverso, Davide 1. May 2017
shutterstock_536684758.jpg
What is Big Data?

Data has always been an important asset in every industry. Since the early days of the information age, business intelligence and descriptive statistics have been used as the standard tools for extracting information and make important decisions from all kinds of collected data. However, as the cost of collecting, storing, and processing data has been dropping exponentially, the amount and the diversity of the data has reached the point where traditional approaches are no longer feasible. In fact, the term Big Data is often used to refer to any data that requires new techniques and tools in order for it to be processed and analyzed.

A more formal definition of Big Data was introduced by Gartner in 2012, in which the well-known 3Vs – Volume, Velocity, and Variety - were used to characterize Big Data. Since then, the 3Vs model has been expanded to several other characteristics, including a fourth V for Veracity and more recently a fifth for Value. Without going into the details of each V, and resisting the temptation to look for a sixth V, we could also look at Big Data from the point of view of the new set of technologies that are helping to solve the challenges in collecting, managing, and analyzing Big Data. These technologies include cloud computing and cluster computing (with Hadoop MapReduce as the most well-known example) for data storage and manipulation, and machine learning for data analysis.

What value does Big Data bring?

The value of Big Data comes from two main use cases: as a source of analytics, and as an enabler of new products and services. In the first use case, big data analytics is used to improve an existing business model by revealing insights from data which was previously too costly to store or process. Amazon’s recommendation system, USPS’s preventive maintenance system, or Walmart’s demand forecasting system are all very good examples. These companies track, collect, and store all available data, from customer transactions to social data, from GPS trails to geographical and meteorological data, then combine them together and use big data analytics to produce high value actionable insights. This would not be possible without the new big data technologies such as cluster computing and machine learning.

In the second use case, big data technologies open up completely new business models and introduce new products and services. Many recent so-called unicorn startups, such as Airbnb, Uber, or Snapdeal, are founded on and enabled by big data analytics. Their products have unique features thanks to the new technologies that they are using. Without following the big data approach, their products would not be able to compete against all the traditional business models.

Big data at eSmart Systems

Since the early days of the company, big data has always been considered as our core value. Big data analytics and machine learning are involved in many of our products.  They are used to forecast electricity demand at substation level, segment customers based on their power consumption patterns, and implement demand response strategies. We have also started to use machine learning to automate the analysis of power lines imaging surveys autonomously conducted by drones with the aim of assisting failure detection and preventive maintenance.

Did you find this article interesting? Subscribe to our blog

Roverso, Davide's photo

By: Roverso, Davide

Davide is Chief Analytics Officer at eSmart Systems and leads the analytics team. He holds a PhD in Computing Science from Aberdeen University (UK). Davide has 25 years’ experience in the field of Machine Learning and Big Data Analytics, with applications in diagnostics, prognostics, condition monitoring, and early fault detection in complex processes, in sectors ranging from energy to medicine and environmental monitoring. He has authored over 90 publications in international journals, conference proceedings and edited books.

  • Write a comment

A blog from eSmart Systems

 

eSmart Systems develops digital intelligence for the energy industry and smart communities. The company is based on more than 20 years of international experience in establishing and operating knowledge based, leading IT and energy related companies targeting global markets.

 

If you want to stay updated on Smart Cities, the rapid development of the energy supply, the power grid and the energy consumption trends, Intelligence First is your water hole.

 

Visit esmartsystems.com