Smart thinking: big data and the energy industry

IBM's Smarter Energy Research Institute spent the last year developing practical, data-driven tools to help transform utility operations. The institute's programme manager David Shepler catches us up on the collaborative projects underway and explains how analytics and predictive modelling could revolutionise asset maintenance, emergency response and renewables integration.


With the influx of big data, the potential of smart grid has shifted dramatically

As the information age continues to bring more sophisticated data innovations with each passing year, the energy industry is starting to catch on in a big way. Utilities are gradually introducing intelligent data analytics to improve their knowledge of core aspects of their business and lay the foundation for the next generation of smarter energy generation, distribution and consumption.

In fact, power utilities' spending on data analytics is increasing every year, helping transform this behind-the-scenes number crunching into one of the industry's biggest growth markets. According to a December 2012 report by market analyst GTM Research, utility spending on data analytics is set to reach $3.8bn by 2020, with US utilities expected to spend as much as $100 per home on grid operations and consumer analytics in the period.

"With the influx of big data, the potential of smart grid has shifted dramatically from the original aim of adding a myriad of new devices toward a complete re-invention of the way utilities do business," said GTM Research president and co-founder Rick Thompson.

"We are now moving into a market where the spotlight will be on the data analytics software that will allow utilities to track, visualise and predict everything from grid operations to electricity consumption."

SERI: a holistic approach to smart energy

"Smarter energy means optimising across the entire energy system."

As the spotlight continues to shine on the potential of 'big data' to transform utility operations, it's unsurprising that IBM Research, the R&D arm of one of the world's largest IT companies, is looking to lead the pack. The company formalised its commitment to bringing new data-driven products to the energy industry in October 2012, when it announced the establishment of the Smarter Energy Research Institute (SERI).

The institute promised to work extensively with three main utility partners - Dutch grid company Alliander, Canada's largest electricity generator Hydro-Québec and Michigan-based utility DTE Energy - to develop practical, information-driven solutions to some of the industry's most persistent problems.

At its core, SERI is about taking a holistic approach to energy data, allowing companies to take a bird's-eye view of their own operations, to gain new insight and even predict problems that might be lurking around the corner.

"Smarter energy means doing everything as efficiently as we can and optimising across the entire energy system," says SERI's programme manager David Shepler, who has been onboard since the foundation of the institute last year.

"That means both finding ways to improve the operations of the utilities on the generation side, through transmission and distribution, and even finding smart ways to help the consumer, the end user of our electric system, do things in better ways."

If this high-level thinking smacks of think tank-style hypothesising, SERI's structure is firmly built for developing concrete, real-world applications that are developed in conjunction with its industry partners. DTE Energy, Hydro-Québec and Alliander have each worked with dedicated IBM teams on two applications that address specific areas of concern.

Participating utilities are investors as well as collaborators, and for Shepler there's no other way to guarantee that the institute's projects are pointed in the right direction.

"We have decades of experience in building analytics and analytic engines," he says. "So we have a number of products that are very powerful in the analytics space that are the tools of the trade. But that can only go so far without the insights of the utilities - what problems do they face on a daily basis; what data do they have? So combining all of that is a very powerful thing, we believe."

Awareness, prediction, optimisation

One application under development at SERI that demonstrates this fusion of expertise is the Outage Prediction and Response Optimization (OPRO) project with DTE Energy, which leverages IBM's existing Deep Thunder weather forecasting software to bring utilities the ability to predict severe weather and its potential effects on critical infrastructure with an unprecedented level of precision. Deep Thunder's processing power translates to a geospatial resolution of 1km x 1km, roughly 12 times more accurate than the weather sources on which the industry currently relies. The project also aims to provide 15-minute updates of key weather parameters, so energy companies can take well-informed steps to prepare ahead of time.

"We've shown in the early prototype that we can reduce reserve margins by up to 16%."

"A really key feature of OPRO is coupling these analytic models together to provide this improved response, from the weather modelling to damage models, which look at historical damage information in the utility and to estimate with higher levels of precision how that weather event is going to translate into damage - downed poles, downed lines, blown transformers and so forth," Shepler says. "And you couple that with things like crew scheduling analytics, for example to pre-position crews so they can respond immediately."

While recent events such as Hurricane Sandy confirm the importance of improving the emergency preparedness of energy infrastructure, other SERI projects are hoping to address the realities of everyday asset maintenance. For example, the Wide-Area Situational Awareness (WASA) application, developed in partnership with Hydro-Québec, is intended to identify grid anomalies and resolve them, before they cause large-scale cascading blackouts that struck India in 2012 or the north-east of the US and southern Canada ten years before that.

"Oftentimes in the transmission grid we're acting reactively, after big problems have already occurred," says Shepler. "WASA is meant to move up the timeline and try to give you some of these early warnings before they turn into large failures."

Integrating renewables

Over and above improvements being made in the fundamental field of grid resiliency, SERI's Wind and Hydro Integrated Stochastic Engine (WhISE) is helping to prove the role that data analytics can play to make renewable energy sources a larger and more reliable part of the energy mix.

"We've shown in the early prototype that we can reduce reserve margins by up to 16%."

As it stands, intermittent generation is a major obstacle to greater renewables integration, as utilities have to build inefficient reserve margins into the generation system to compensate for the variation of supply.

The industry is working on smart grid and demand response technologies to help handle intermittency, but as Shepler explains, WhISE is making some impressive headway on the problem with the comparatively easy-to-implement method of mathematical modelling.

"We believe that by explicitly modelling that variability into the unit commitment equation, we can reduce these margins considerably. We've shown in the early prototype that we can reduce the margins by up to 16%, and this is by just using really advanced mathematic techniques, and making the assumptions in the model explicit, rather than just doing it in this crude way of having very large margins."

Modelling the future

It's been a year since IBM founded SERI, and the institute has clearly wasted no time coming up with an impressive number of data-driven innovations for the energy industry. SERI held its first conference in June, bringing the IBM staff and partner utilities together for the first time. Shepler is effusive in his praise for the event and the opportunity for SERI's industry members to share their thoughts with other energy companies considering joining the institute. It also gave him hope that SERI is taking the right steps at the right time for the wider energy sector.

"We wouldn't be doing this if we didn't believe the market is ready for us," he says. "We have a lot of work to do and that's why we have the institute, to solve these problems and to create these analytics in a way that will translate to many utilities. The long-term goal is to create real capabilities that will be adopted widely in the industry."

Of course, the bulk of convincing the power industry to adopt big data as a core practice is providing solid evidence that these innovations will lead to significant operational advantages and, therefore, financial savings. All of SERI's programmes incorporate 'business value' targets that researchers will be aiming to reach and validate, to create compelling business cases. In the case of OPRO, for example, the aim is to reduce DTE Energy's outage restoration costs by ten to 20%.

Despite the goals IBM hopes to achieve under SERI's banner, Shepler recognises the institute's work is really a continuation of a data analytics trend that is already well underway.

"It's not a hard sell, these days. [Utilities] have all these siloed systems; each of those is a wealth of data, and they recognise that there some compelling problems they could solve by bringing that data together. The challenge is to show them that it's real, that it's not just an abstract notion that sounds interesting, but that we can really do it."

Further work to validate SERI's applications is necessary, and IBM plans to start by initially rolling out "a few hard-hitting capabilities" that can be built on in the coming years. "I would tell you that in a year or less you'll begin to see the beginnings of this," Shepler reveals. "We'd like the institute to continue to exist for many years to come, and we imagine these capabilities evolving with it."

It will clearly take time for the energy industry to fully embrace the potential of data analytics, and the day when applications like WASA are common practice for every utility is likely years off. But to paraphrase a famous quote, SERI's work shows that while the end is not yet in sight for bringing big data to the utilities sector, nor even the beginning of the end, 2013 might well mark the end of the beginning.

Follow Chris Lo on Google+

Energy group link

Related content


Demand response roll out - saving power at peak times

Demand response, which reduces the use of energy at peak times, is the second step, after smart meters, of the smart energy revolution.

Reliable renewables with cryogenic energy storage

Intermittent supply is an ongoing problem for the development of the renewable power market.