The good, the bad and the ugly: Real life stories of end-user engagement evaluation in energy efficiency projects

Panel: 4. Monitoring and evaluation for a wise, just and inclusive transition

Authors:
Richard Bull, Nottingham Trent University, United Kingdom
Ashley Morton, De Montfort University, United Kingdom
Andrew Reeves, De Montfort University, United Kingdom

Abstract

Evaluating engagement for energy efficiency projects often uncovers barriers around end-users’ understanding and acceptance, or issues of trust surrounding interventions, technological advances, and new policies. Monitoring and evaluating end-user engagement is increasingly at the forefront of policy agendas and research funding calls, but there is often a lack of transparency when reporting the true challenges and failures around engagement in energy efficiency projects. With ambitious targets across Europe for energy efficiency improvements there is a need for incorporating lessons learnt into project development instead of attempting to redesign the wheel over and over.

This paper adopts a storytelling approach reflecting on three EU and UK research projects that the authors have been involved in. We unpick the real-life stories of implementing and evaluating different energy efficiency projects that were reliant on end-user engagement. Storytelling signifies a broad understanding of reflecting in a more personal capacity. Underpinning this approach is what Janda and Tapouzi call a ‘hero story’ perspective that presents an ‘airbrushed,’ edited and overly optimistic view of the projects, for example, exaggerating efficiency savings. The alternative they suggest is a learning story that acknowledges ‘the specific context, location and people involved’ in order to provide greater impact and a more realistic picture of the ‘real world’. This fits squarely in the interpretivist approach to knowledge with parallels to narrative autoethnography that draws on and explicitly acknowledges the personal role, experience and involvement of the researcher in the projects (Murphy 1999, Berger 2004).

Our three case studies will be explored through the lens of context, location and people with a spirit of honesty, vulnerability and openness. This should reveal insights in order to help others in similar positions perhaps feeling like failed heroes and offers recommendations for project design and evaluation ensuring that end-user engagement is more wisely and deployed to support future sustainable energy transitions.

Our projects range from small scale internal projects in our home University to large consortium EU H2020 projects. Underpinning all of these projects has been focus on public participation, i.e. ensuring that the design of any ICT based solution was underpinned and subjected to scrutiny by those who would be using the tool. We focus on three.

Storytelling allows the richness of the project to come to the fore. All three projects had two features in common. They were all developing a technical ICT based solution to visualise energy consumption and were ambitious in undertaking substantial public engagement in the form of workshops with the building users. Two of out these three projects encountered substantial challenges around implementation and delivery. Notably these challenges were never around the technical development of the ICT tools. That’s not to say that there were no technical difficulties in implementing monitoring and measuring equipment, or in designing an appropriate interface in the time allocated.

However, in all instances it was challenges around the social, political or institutional environmental factors that affected the public engagement or adoption of the tool and ultimately the success. Redundancy, relationship breakdown and insecurities around roles and personalities, poor project management and finally a global pandemic have all impacted on the delivery and adoption of the projects.

Yet these are often the very features that are removed from the analysis as we focus in evaluating and defining projects purely in terms of the success in energy saving. Failure to engage with and talk about the real world interpersonal and social challenges and barriers affecting the adoption of energy savings tools will mean mistakes will be repeated and institutions will fail to meet their low carbon ambitions.

Watch presentation