Attribution Methodology Wars: Self-Report Methods versus Statistical Number Crunching - Which Should Win?

Panel: Commercial Buildings: Program Design, Implementation, and Evaluation

Author:
Gay Cook, Summit Blue Canada Inc.

Abstract

Several methods have been used to assess the impact of free riders and spillover on both residential and nonresidential energy efficiency programs. Those methods include self-reported program response, econometric methods such as billing analysis, econometric or discrete choice models, and market sales based methods such as saturation data analysis. Each approach has benefits and drawbacks, each is appropriate in different circumstances, and each has its own advocates. However, it is often hard to know which approach is best for a given program. How are program evaluators to know which approach they should use in what circumstance? What are policy makers to believe when told that "this method is the best?" How can these approaches be placed in context and be assessed objectively?

This paper explores these fundamental questions and provides the background to place the approaches in a broader perspective. It describes the various approaches and presents their pros and cons, and also provides some practical examples of how to determine which approach is most suited to specific circumstances. The paper is based on free ridership and spillover research done on a variety of programs in the United States and Canada and a thorough literature review.