This is the final post in the Predicting Global Energy Demand using Spark series. The series has the following posts.

Problem:

Assuming there was full day of outage, Calculate the Revenue Loss for a particular day next year by finding the Average Revenue per day (ARPD) of the household(Use the below tariff plan).

Time of Use (TOU) tariff plan:

Brief Solution:

We need to find the average cost per day in a year which will be the revenue loss.
To solve this problem we need to aggregate the data on hourly basis. Based on the tariff plan calculate the cost per hour. Aggregate this cost into daily basis. Add the cost for every day in a year and get the average by dividing it with the total number of days in that year. This average represents the revenue loss in case of outage.

Getting per hour revenue:

To get the revenue per hour you can refer part 3 hourly aggregation. After getting the hourly data submeter readings will be multiplied by tariff applied for that particular hour. This will give us the cost on every hour.

def getPerHourRevenue(inputRDD:RDD[Record]):RDD[Record] = {
    val dataAggregator = new DataAggregator()
    val hourlyRDD = dataAggregator.hourlyAggregator(inputRDD)
    hourlyRDD.map(value => {
      var ((date, hour), record) = value
      val hourOfDay = record.hourofDay
      if ((hourOfDay >= 0 && hourOfDay <= 4) || (hourOfDay >= 10 && hourOfDay <= 15)) {
        record.totalCost = (record.subMetering1 * 4 + record.subMetering2 * 4 + record.subMetering3 * 4) / 1000
      } else if ((hourOfDay >= 5 && hourOfDay <= 6) || (hourOfDay >= 16 && hourOfDay <= 19) || (hourOfDay >= 22 && hourOfDay <= 23)) {
        record.totalCost = (record.subMetering1 * 6 + record.subMetering2 * 6 + record.subMetering3 * 6) / 1000
      } else if (hourOfDay >= 7 && hourOfDay <= 9) {
        record.totalCost = (record.subMetering1 * 12 + record.subMetering2 * 12 + record.subMetering3 * 12) / 1000
      } else {
        record.totalCost = (record.subMetering1 * 10 + record.subMetering2 * 10 + record.subMetering3 * 10) / 1000
      }

      record
    })
  }

Then this hourly data is aggregated into daily basis for which you can refer the code dailyAggregator in the part 2. Get the average cost for a day from this daily aggregated data which represents the revenue loss in the case of outage.

def getRevenueLoss(inputRDD:RDD[Record]):Double = {
    val dataAggregator = new DataAggregator()
    val revenueRDD= getPerHourRevenue(inputRDD)
    val dailyRDD = dataAggregator.dailyAggregator(revenueRDD)
    val totalCostCalcRDD = dailyRDD.map(value => ("sum", value._2.totalCost)).reduceByKey((a, b) => a + b)
    val revenueLossForDayOutage = totalCostCalcRDD.first()._2 / dailyRDD.count()
    revenueLossForDayOutage
  }

So this will end the explanation of the solutions for problem given in hackathon. Each one of us from the team took a particular problem and explained the solution to it. Hope you go through all the solution and get the most out of it.

You can refer here for complete code.