Using program logic model analysis to evaluate & better deliver what works

Lori Megdal, Megdal & Associates
Victoria Engle and Larry Pakenas, the New York State Energy Research and Development Authority
Scott Albert, GDS Associates, Inc.
Jane Peters, Research Into Acton, Inc.
Gretchen Jordan, Sandia National Laboratory

Keywords

evaluation, logic models, program theory, market transformation, program redesign

Abstract

There is a rich history in using program theories and logic models (PT/LM) for evaluation, monitoring, and program refinement in a variety of fields, such as health care, social and education programs. The use of these tools to evaluate and improve energy efficiency programs has been growing over the last 5-7 years. This paper provides an overview of the state-of-the-art methods of logic model development, with analysis that significantly contributed to:

Assessing the logic behind how the program expects to be able to meets its ultimate goals, including the "who", the "how", and through what mechanism. In doing so, gaps and questions that still need to be addressed can be identified.

Identifying and prioritize the indicators that should be measured to evaluate the program and program theory.

Determining key researchable questions that need to be answered by evaluation/research, to assess whether the mechanism assumed to cause the changes in actions, attitudes, behaviours, and business practices is workable and efficient. Also will assess the validity in the program logic and the likelihood that the program can accomplish its ultimate goals.

Incorporating analysis of prior like programs and social science theories in a framework to identify opportunities for potential program refinements.

The paper provides an overview of the tools, techniques and references, and uses as example the energy efficiency program analysis conducted for the New York State Energy Research and Development Authority's (NYSERDA) New York ENERGY $MART SM programs.