The literature has significantly acknowledged the importance of involving end users and multiple stakeholders in a living lab. End users in living lab studies are perceived as co-designers who actively participate in developing products and services ^[1][2][23,24]. A stakeholder is described as an individual or group that is affected or potentially affected by a particular decision or development ^[3][28]. With this perspective, Dvarioniene et al. ^[4][29] p. 514, define stakeholder participation as “a process of involving everybody who takes an interest (or “stake”) in a project to foster its acceptance, get contribution and support as well as manage possible conflicts and oppositions”.

The stakeholders involved in living labs outside the energy sector were mentioned in 11 of the sampled studies (see Table 16). These studies fall within the sectors of health, education, urban farming, computer and social sciences, and ICT. The stakeholders identified include: (i) private companies and third sector organisations; (ii) researchers; (iii) financiers; (iv) project managers; (v) pilot and panel managers; (vi) business managers; (vii) users; (viii) industry representatives; and (ix) public sector authorities.

^11][41].

In practice, the role of the user was therefore primarily described as an informant during the conceptual phase, an experimenter or tester of existing technology during the implementation phase, and a contributor/collaborator influencing designs in the evaluation of existing technology or systems ^{[2][3][11][12]}[24,28,38,41]. In a city-wide living lab environment, poor user involvement is attributed to the inability to define who the user is and the lack of financial resources ^[2][24].

The stakeholders involved in living labs in the energy sector are mentioned in 14 of the sampled papers and include the following: (i) financiers; (ii) public entities; (iii) researchers; (iv) SMEs; (v) large private businesses; (vi) trainees, teachers, and students; (vii) administrative personnel; (viii) users; and (ix) industry experts (Table 27).

The most common stakeholders in the sampled studies outside the energy sector are the users, industry representatives and researchers. Furthermore, research institutes ^[1][2][13][23,24,34] at times partner with non-profit organisations ^[9][10][43,50]. Seldomly, municipal authorities ^[5][30] are the predominant initiators of living lab projects in these papers.

Apart from providing financial resources for technological innovations and research in living labs, the role of a financier is also to consistently evaluate the progress of the living lab ^[2][3][24,28]. Like the project manager, who has the role of overseer, living lab financiers are present from the initiation through the implementation and evaluation phases. However, they are not actively engaged in living lab processes on the ground. Evidence highlighting industry representatives comprises designers, manufacturers, technology providers ^[8][37], and entrepreneurs ^[5][30]. These actors are involved in designing, developing, and implementing technical solutions and providing general technical support or management in a living lab.

In an urban living lab initiated by the municipality, the private sector and third sector organisations adopted the role of being an intermediary that provides a communication platform between municipal authorities and civil society to express their goals and needs as well as being a coordinator that gathers all human and financial resources required to achieve the objectives of a living lab initiative ^[5][30]. Reviewing a ‘smart city living lab process’, Ståhlbröst et al. ^[3][28] further highlight the involvement of a pilot manager who is involved in setting up pilot projects within the living lab, a panel manager who primarily recruits and interacts with users, and a business manager who is responsible for the commercialisation of technology.

Regarding researchers as stakeholders, these actors are shown to be either from private research institutes or public universities and are sometimes identified as participatory design or action researchers, students, and trainees ^[3][5][9][28,30,50]. Researchers are involved in prestudies to discover the needs and stories of potential users, collect data during the testing phases, develop concepts, and support other methodological processes in the living lab ^[2][11][24,41].

Public sector authorities were identified as city councils and building managers. They contribute knowledge that provides the context of a problem in a particular area, similar to what users do during the need-finding phase. Users (mostly referred to as ‘end users’ or ‘participants’) differ according to the context of the living lab. For instance, in private home settings, they are noted as residents ^[5][30] or households ^[12][38], while in a medical context, they are described as ‘elderly people’ ^[11][41]. Ståhlbröst et al. ^[3][28] add that end users, municipal authorities, and building managers are stakeholders who have a moral claim on the living lab and are the owners of problems embedded in that place.

In theory, users are depicted as actual cocreators; however, this has been proven to be limited in practice. Leminen et al. ^[7][3] state that within a living lab, a user may adopt the role of being an informant, a contributor, a tester and/or a cocreator. A cocreator is described as an inexperienced technological designer who, without being made aware of the possibilities, may not accurately articulate what they want during the design processes. Hence, they work with other (‘expert’) stakeholders to design, develop, and produce tangible solutions in a mutual learning environment ^[

The prevalent stakeholders are the users, industry experts, researchers and public authorities. Moreover, partnerships between municipalities, universities (or research institutions), and private companies ^{[16][19][20][22]}[26,32,36,40] are the main initiators of living lab projects relating to energy in the sampled studies.

The role of public entities has been stated in five studies ^{[16][22][23][24][25]}[20,26,36,47,52]. These entities include actors such as state-owned energy companies and municipal authorities that provide a regulatory and legal framework for the living lab and incentivise SMEs to implement pilot projects. The financier plays a role in enabling operations within the living lab by contributing financial resources ^[18][20][25,32]. In Ståhlbröst ^[18][25], the SME actor is represented as the owner of energy technology that is tested in the living lab within private homes and can thus be seen as playing the role of a living lab developer or initiator. In contrast, private businesses, also referred to as energy market actors in Krogstie et al. ^[23][20], are seen as technology designers who are also at liberty to request funding from project initiators to develop energy products.

Another stakeholder group identified in energy living labs in six papers is industry experts, which include architects, craftspeople ^[25][52], hardware developers ^[18][25], and suppliers of technology ^[23][20]. This group provides industry input about techniques and relevant skills, tests products and systems in the living lab, assists in research analysis, and participates in brainstorming sessions with users and other stakeholders to come up with solutions to energy problems.

Researchers from research institutions contribute scientific and evidence-based knowledge to support decision-making and help realign testing strategies according to the living lab’s regulatory requirements, objectives, and resources ^[25][52]. Furthermore, in writing about the first energy living lab in the United Kingdom, Sovacool et al. ^[17][51] illustrate how researchers are the actors who primarily interact with end users in the living lab, from interviews before the installation of energy systems to surveys relating to users’ satisfaction with the system after installation and use. Actors such as trainees, teachers and students assist in data collection processes and partake in vocational training ^[22][36]. The administrator is responsible for sending out emails concerning living lab activities and engagements to other stakeholders ^[26][49].

The end users in energy living labs, commonly referred to as ‘participants’, are actors such as homeowners, residents, building occupants, and managers ^[14][17][22,51]. Their roles are elaborated in 9 of the 14 energy living lab papers, which allude to the end users’ involvement in various research studies and surveys as consumers of technology as well as their participation in the implementation and testing of installed software and energy devices such as smart metres to assess their effectiveness ^[15][16][21,26]. Similar to user roles observed in living labs outside the energy context described in Section 4.1, end users in operational energy living labs can be seen to serve as informants, contributors/collaborators and testers for evaluating existing energy innovations to increase usability rather than as actual cocreators of tangible solutions.

There is limited evidence highlighting how stakeholders are identified in a living lab project. Only 2 out of 35 sampled studies ^[4][24][29,47] provided information on how stakeholders were identified. Stakeholders can be identified through the mind-mapping method or reverse mind mapping, done individually or during a group brainstorming session. In the two studies by Dvarioniene et al. ^[4][29] and Giannouli et al. ^[24][47], the mapping of stakeholders was used to generate a list of actors involved in a specific sector (i.e., energy), persons who are considerably affected by a decision or project, or diverse interest groups based on their skills, economic, and political interests or knowledge ^[4][24][29,47]. A visual matrix that clusters stakeholders according to their importance, influence, and power during planning or later implementation may also be drawn from the stakeholder list. In an energy lab context for enabling energy-conscious communities, Dvarioniene et al. ^[4][29] further allude to an in-depth stakeholder analysis that claims to be the best method for obtaining a well-represented final list of stakeholders. The authors assert that this simplifies how and when the stakeholders can be involved and helps determine their position or role in the project. The main issues to be considered in the deeper analysis of stakeholders are summarised in Table 38.

Literature explaining the recruitment process of stakeholders and end users involved in a living lab is limited. Nonetheless, cold-call techniques that involve sending out email newsletters that call for participation, individually contacting potential stakeholders via telephone, and attending subject-related events to engage with relevant persons are highlighted in one study ^[10][43].

End-user recruitment in research to enhance the design interface of smart home technology began by first publicising the living lab study via local radio stations and newspapers ^[12][38]. Participants were thus selected when they fulfilled the criteria set by the researchers, which were done through an online portal and later through telephone interviews. Households had to fall within a particular postcode, had to have a stable internet connection, had to provide adequate reasons for wishing to participate, and had to express their project expectations and their knowledge of products and hard-/software prototypes ^[12][38].

Stakeholder recruitment processes in most papers concerning living labs in the energy sector are vague. The papers state that end users were either recruited voluntarily ^[18][25] or invited to test and evaluate technology ^[16][26], subject to their interest in energy solutions ^[22][23][20,36]. In Jahn et al. ^[14][22], end users as building occupants seemed not formally recruited but rather participated in the living lab study because operational changes were occurring within their living space. Only Sovacool et al. ^[17][51] elaborate on the process of recruitment of end users, which was through the use of telephone interviews. Further, the user recruitment in Sovacool et al. ^[17][51] energy living lab was based on a screening process that included households with specific technological infrastructure, permanent residents, and people possessing knowledge about smart energy technologies.

Evidence indicating the operations of engagement amongst various stakeholders and end users involved in a living lab is elaborated in 12 of the 35 studies. Researchers, technology developers, and end users mostly engage directly with each other through a series of face-to-face interviews ^[21][46], telephone interviews ^[17][18][25,51], user-pool brainstorming workshops ^[23][20], kick-off events ^[12][38], questionnaires ^[26][49], roundtable discussions ^[5][30], and technical workshops ^[15][21]. The latter engagements are utilised as instruments for data collection for researchers and developers to enhance their contextual understanding of users and their needs during the early phases of the living lab, inform users about the research programme, and train users on how to install smart plugs.

The literature shows that after the setup and installation phase and during the final technology or system development phase, end users can exchange their immediate experiences with the user community, convey problems, and receive solutions from researchers via online discussion forums, instant messenger groups, and regular phone calls ^[12][18][25,38]. This encourages ongoing dialogue and cultivates close relationships between users and researchers. Overall, engagements between all participating stakeholders in a living lab ensue through regular meetings ^[9][50], occasional technical and codesign workshops, conferences ^[4][19][29,40], and focus groups ^[5][20][30,32]. This is where certain stakeholders present the plans and progress of the living lab, and the codesign of technological and scientific solutions amongst expert stakeholders and municipal officials occurs ^[25][26][49,52].

Keeping stakeholders and end users engaged throughout the living lab process is one of the obstacles to collaboration. Some partners either become passive in their participation due to undefined roles and expectations, or the level of commitment decreases, resulting in their absence from certain activities ^[9][20][32,50]. In the case of end users, valuable quantitative data were lost due to household renovations, specific inhabitants travelling or selling their homes while the researchtudy was in progress, and users not providing feedback ^[2][21][24,46].

In the literature, there is a gap regarding gendered considerations in living labs in the energy context. The power relations of gender in innovation and design decision-making processes in the IT and ICT industries were, however, highlighted by three studies ^[2][9][11][24,41,50].