A process to develop operational bottom-up evaluation methods – from reference guidebooks to a practical culture of evaluation

Jean-Sébastien BROC and Bernard BOURGES, Ecole des Mines de Nantes (DSEE), France
Jérôme ADNOT, Ecole des Mines de Paris (Centre of Energy and Processes), France

Keywords

evaluation methodology, bottom-up method, impact evaluation, theory-based evaluation, Energy Services Directive, experience capitalisation, evaluation capacity building

Abstract

Needs for evaluating energy efficiency (EE) activities are increasing, for the accounting of results and for understanding their success/failures. Indeed evaluation results should be used for both reporting past activities and improving future operations. Lack of easy to use methods is pointed out by local stakeholders as a major barrier to evaluation. Another issue is the frequent negative perception of evaluation, experienced as a control and/or a waste of time.

This paper presents a systematic process to develop bottom-up evaluation methods designed to fit to stakeholders needs: directly operational, easy to appropriate, providing useful conclusions to improve operations and to communicate about their results.

Our approach relies on the principle of experience capitalisation and on an organisation with two levels, central and on-field. It aims to create conditions for continuous improvement. Moreover it should insure involved stakeholders do actually take part in and take advantage of the evaluation process.

This methodology handles both impact and process evaluation. For the impacts, focus is on calculations transparency, data quality and reliability of the results. Regarding operation process, main issues are analysing causality between actions and results, and detecting the success and failure factors.

This work was first developed for the evaluation of local operations in France. The resulting methodology was tested on two case studies from the Eco Energy Plan, a local EE programme implemented in South-East of France.