The theory-based policy evaluation method applied to the ex-post evaluation of climate change policies in the built environment in the Netherlands

Mirjam Harmelink, Suzanne Joosen, and Kornelis Blok, Ecofys

Keywords

ex-post evaluation, energy efficiency, climate, policies, built environment, theory-based evaluation, program theory, program logic model

Abstract

The challenge within ex-post policy evaluation research is to unravel the whole policy process and evaluate the effect and effectiveness of the different steps. Through this unravelling of the policy implementation process, insight is gained on where something went wrong in the process of policy design and implementation and where the keys are for improving the effectiveness and efficiency.

This article presents the results of an ex-post policy evaluation of the effect and effectiveness of the Energy Premium Regulation scheme and the Long Term Voluntary agreements to reduce CO2 emissions in the built environment in the Netherlands applying the theory-based policy evaluation method. The article starts with a description of the theory-based policy evaluation method. The method begins with the formulation of a program theory, which describes the 'ideal' operation of a policy instrument, from the viewpoint of the policy makers. Thereupon the theory is checked and adapted through interviews with policy makers and executors, and the cause and effect chain is finally translated to (quantitative) indicators. The article shows that the theory-based evaluation method has benefits over other ex-post evaluation methods that include:

The whole policy implementation process is evaluated and the focus is not just on the 'end-result' (i.e. efficiency improvement and CO2 emission reduction).
Through the development of indicators for each step in the implementation process, "successes and failures" are quantified to the greatest possible extent.

By applying this approach we not only learn whether policies are successful or not, but also why they succeeded or failed and how they can be improved.