This is the first post in the Predicting Global Energy Demand using Spark series. The series has the following posts.

This series of blogs will walk you through our complete solution , designed and implemented for predicting future energy demand using spark.

This problem was solved within 24hrs in hackathon.

Problem statement

Predict the global energy demand for next year using the energy usage data available for last four years, in order to enable utility companies effectively handle the energy demand.

Fluctuating energy demand for utilities is becoming a very big problem. To be more precise about the problem, the demand for energy over a period of time is not consistent. This in turn becomes huge bane for the utilities with varying energy demands. As we know with the existing systems storing of electricity is extremely inefficient, this will increase difficulties for the utilities to bank energy against a time of sudden demand.

As we observe, energy demand rises and falls throughout the day in response to a number of things,including time and environmental factors.The difference between the demand in extremes is important, because utilities must be able to handle demand with supply. One approach to managing electricity demand is building more generation facilities that can be brought online to manage peaks. But using right estimations allows companies to make better decisions.

Available data

Data for energy consumption from 2008-2012 is available here.

The following is the structure of data

  • date: Date in format dd/mm/yyyy
  • time: time in format hh:mm:ss
  • global_active_power: household global minute-averaged active power (in kilowatt)
  • global_reactive_power: household global minute-averaged reactive power (in kilowatt)
  • Voltage: minute-averaged voltage (in volt)
  • global_intensity: household global minute-averaged current intensity (in ampere)
  • sub_metering_1: energy sub-metering No. 1 (in watt-hour of active energy). It corresponds to the kitchen, containing mainly a dishwasher, an oven and a microwave (hot plates are not electric but gas powered)
  • sub_metering_2: energy sub-metering No. 2 (in watt-hour of active energy). It corresponds to the laundry room, containing a washing-machine, a tumble-drier, a refrigerator and a light.
  • Sub_metering_3: energy sub-metering No. 3 (in watt-hour of active energy). It corresponds to an electric water-heater and an air-conditioner

Problems we addressed

  • What would be the energy consumption for the next day?
  • What would be week wise Energy consumption for the next one year?
  • What would be household’s peak time load (Peak time is between 7 AM to 10 AM) for the next month.
    During Weekdays
    During Weekends
  • What are the patterns in voltage fluctuations?
  • Can you identify the device usage patterns?
  • Assuming there was full day of outage, Calculate the Revenue Loss for a particular day next year by finding the Average Revenue per day (ARPD) of the household using given tariff plan

In upcoming blog posts , we will be sharing our solution to above problems.

Leave a reply