The California Evaluation Framework: Moving Evaluation into a Systems Approach for Planning and Conducting Energy Program Evaluations

Marian Brown, Southern California Edison Company
Lori Megdal, Megdal & Associates
Pete Jacobs, Architectural Energy Corporation
Nick Hall, Johna Roth, TecMarket Works

Keywords

Abstract

The California Public Utilities Commission has ordered that a new framework be established to guide the planning and conduct of California’s energy efficiency program evaluations. A primary goal of the new framework is to establish an evaluation approach that provides reliable information to help ensure meeting California’s energy needs, while also supporting continued program improvements and helping to meet the information needs of policymakers and program managers.

The new Framework provides a systems approach to planning, conducting and funding evaluations of energy programs, instead of the more traditional program-specific or sectorspecific approach. The Framework provides a structured decision process in which portfoliolevel considerations of quality, uncertainty, and reliability directly influence decisions about what type(s) of evaluation should be conducted, when the evaluations should be conducted, and the approaches appropriate for conducting the research. It considers evaluation designs for the following types of evaluations: impact (both gross and net), metering and monitoring efforts, process, market effects studies, non-energy effects research, and information and education program evaluations. Each chapter summarizes alternative approaches and their strengths and weaknesses. The Framework also includes chapters on appropriate sample design and analysis, on identifying and dealing with sources of uncertainty, and on a market-based perspective for calculating avoided costs and conducting cost-effectiveness tests.

This paper summarizes the contents of the Framework.