Evaluating Attribution, Causality, NEBs, and Cost Effectiveness in Multifamily Programs: Enhanced Techniques

Jody Myers and Lisa A. Skumatz, Skumatz Economic Research Associates, Inc.

Keywords

Abstract

As part of several evaluation projects, the authors have developed a number of useful steps to improve the attribution of impacts to program interventions. This paper presents an early application of enhanced evaluation methods to the multi-family sector – illustrating attribution/causality work for a multifamily new construction program. The methods and results are presented in this paper, and include:

Theory-based evaluation, indicators, and hypothesis testing;
Estimation of gross impacts and attribution of impacts using distribution and ranges of measure and intervention impacts, rather than less reliable point estimates. This uses more of the information gathered during data collection, provides a band of confidence around results, and more fully reflects the range of impacts induced by the programs.
Enhanced measurement of naturally occurring adoption, multiple types of spillover/indirect market effects, and other impacts.
Examination of the results across program types to explore patterns and important differences.
Cost effectiveness assessment.

These extra steps improved the reliability and robustness of the results of the causality analysis and provided a better foundation to guide benefit-cost analysis and program and investment decisions – an important goal of an evaluation. The paper highlights benefits and impact of these approaches. We use the data to estimate the total net program impacts including net energy impact and net non-energy benefits (NNEB) for the program – information that was used in estimating the program’s cost-effectiveness.