Evaluation, Measurement and Verification (EM&V) is a vital part of the utility program life cycle; however, many utilities don't do enough Evaluation before starting a new program for their customers. Many third-party vendors have very slick sales presentations that can over promise program results. Here we'll discuss a few things you should consider before launching the program, including penetration rate, cost per measure, effectiveness, and program costs.
Penetration Rate
One of the most important factors to consider when launching a new program is the likely penetration rate, or adoption rate. That is, what percentage of your customers who could participate in this program will participate. It doesn't matter how effective the measures are if none of your customers will take part. For example, with EV load management, smart charger and other hardware programs struggle to reach more than 10-20% of customers. This means that even if 100% of participating customers achieve the desired outcomes or behaviors, 80-90% of your eligible customers are not. The same can be said for voluntary time of use rates, where most utilities struggle to achieve even 1% adoption.
Cost per measure and real world use
Is your utility paying more for programs than the value you earn? In the northern half of the US, some utilities have chosen to incentivize cold climate heat pumps. However, our data shows that ductless systems (as opposed to central ducted systems) aren't actually used for heating. Utility rebates have been designed with added kWh sales from winter heating, and without that usage, the rebate isn't recouped as expected. There is a similar situation with peak-reduction efforts, where the cost of getting customers to enroll and participate can often be greater than the savings from reducing demand and transmission costs.
Effectiveness
Related to real world use is effectiveness. Does the program or measure perform as well as advertised. Thinking again about hardware controls, for EVs, HVAC or hot water heaters, are those devices reachable 100% of the time? Analysis for our clients indicates that between 10 and 30% of hardware devices are non-functional for one reason or another at any time. Do your program calculations include hardware issues?
Program Costs
The final, and perhaps most important consideration before launching a program are the costs of running the program yourself versus hiring an outside vendor. Monthly program costs, either from fees, software access or actual hours spent by your staff managing the program can eat away at any benefits from your programs.
How to move forward?
How can utilities protect themselves from overly expensive programs? This formula could help:
Value = Penetration rate x Effectiveness x Value (per kWh or kW, also consider seasonality and time of day differences based on your specific economics)
Total Cost = Vendor costs + hardware/software costs + marketing + rebates
Total value = Value - Total Cost.
Start with the information provided by your vendor or internal teams, but then be sure to calculate a Best,Worst and Average case scenarios, to see what the real effectiveness could be. Even small changes to the initial conditions can lead to drastically different outcomes, particularly from marketing penetration and real world effectiveness.
Of course, if your utility is overwhelmed by the process of evaluating programs, give us a call, and we'll be happy to have a conversation.
