Serious Games and Co-Creation: History
View Latest Version
Please note this is an old version of this entry, which may differ significantly from the current revision.
Contributor:
Merit Tatar
,
Mariia Khrapunenko
,
Richard Karl Henahan
,
Andra Asser

In the gaming industry, serious games as a tool to share educational content and provide learning experiences have gained popularity in recent years by leveraging the power of games for purposes beyond pure entertainment. Unlike other services, products, or solutions that usually serve a single purpose, a serious game is tasked with not only being fun and engaging but also raising awareness and educating the player.

  • co-creation
  • co-design
  • design thinking
  • serious games
  • serious game design
  • participatory game prototyping

1. Serious Games

Even though there is already much literature available that discusses serious games [1][2][3], their definition remains broad and is mainly interpreted in relation to their aim and purpose. Since the time serious games came to prominence in the 1970s, their main characteristic has been the educational component, which was combined with entertaining aspects to train and/or develop the skills of the players [4]. Tuli and Mantri [5] concisely define serious games as ones that are developed and designed for aims other than entertainment, possessing immersive and interactive features with problem-solving approaches to improve teaching and learning.
Serious games can be effective tools for promoting behavioural change and raising awareness about several topics like social, health, and environmental issues [1][6] and often focus on challenging players with complex problems and decision-making scenarios. The common notion about serious games is that they facilitate an environment that allows the player to experiment and learn through engagement in hypothetical real-world situations [7][8]. Serious games act as a good basis for decision-making and communication, and many of them allow the players to act as a particular character and interact with each other within a regulated setting to achieve the set goals and objectives [1][9][10]. Lanezki et al. [11] add that serious games motivate players to reach goals and learn cooperatively, while at the same time implementing complex issues into the easy-to-understand game environment and raising awareness about them. Thus, they engage players in meaningful experiences, encouraging them to adopt new behaviours, understand complex problems, and make informed decisions (in [1]).
Moreover, serious games, especially digital ones, can serve as effective platforms for research and data collection [2][12]. By designing games that simulate real-world scenarios, researchers can gather data on performance, user behaviour and preferences. This information aids in understanding human interactions and refining further strategies in diverse domains.
The main difference between serious games and regular digital games lies in their end goals. Whereas regular digital games aim to keep the player engaged in the game as long as possible through fun and enjoyable gameplay, the serious game design is more aimed at the learning outcomes and experience with the knowledge that the player will get. The gameplay in serious games is also important, but plays a secondary role compared with educational aspects [13][14][15]. Nevertheless, for serious game design, it is usually a challenge to integrate learning principles in the gameplay without losing its entertainment factor and motivation to play the game further [13]. Thus, it can be argued that the more developers can incorporate the expectations of the potential users on all aspects into such game design, the more relevant is the result and that is where participatory design process potentially adds value.
With technological advancement, particularly digitalisation, the potential role serious games can play is becoming increasingly relevant [16]. The integration of digital tools in education has become even more relevant due to the COVID-19 pandemic, forcing teachers and students to adopt to digital and remote learning [17]. Paired with the fact that schoolchildren spend more time on game-related activities due to the massive increase in mobile devices and game consoles, increases the relevance of serious games concept even more [18]. Serious games can also be applied to science and research, which is well illustrated in the Dekker and Williams literature review on the use of serious games being co-designed with their target group to address depression and anxiety disorders [19]. Taken together, the use of serious games is highly relevant and topical considering the advancement of digital tools—particularly as educational tools—over the past few years, which underscores its significance in a digital society.

2. Evaluating Serious Games

Serious games have proven to be effective when used in various contexts because of their ability to teach, persuade, and entertain [20]. Their effectiveness has been measured through the experience of the players: their increase in literacy; raised awareness about the issue in focus; acquired skills and knowledge, etc. [7] According to Wainess and Oneil [21], the effectiveness of a serious game can be defined in terms of (1) the intensity and longevity of engagement with a game, (2) the commercial success of a game, and (3) the acquisition of knowledge and skills as a result of the implementation of a game as an instructional medium.
There has been research carried out to evaluate the effectiveness of serious games in particular fields to see what further improvements could be made in developing such games. For instance, Wu et al. [22] studied twenty-one serious games related to the area of energy consumption. The results of their study show that a serious game can be a valuable tool in altering energy consumption behaviour for consumers. Additionally, by incorporating a well-designed gameplay concept, a serious game environment can introduce ground-breaking ways of interaction and relationships among consumers. Hammady and Arnab’s [1] review of over 200 articles about serious games in different fields aimed to identify game design mechanics and features that are reported to commonly influence behaviour change during and/or after the interventions through these games, and Tan and Nurul-Ansa [23] reviewed the reasons for the increasing popularity of serious games, the features of successful serious games, and current trends in serious game application in environmental education, stressing their potential to “facilitate interaction between learners and the natural environment, and in turn strengthen environmental awareness and appreciation” (p. 19). Indeed, several research findings consistently highlight the effectiveness of serious gaming as a powerful tool for enhancing learning outcomes, and when compared to traditional methods (see also [1]). That is why serious games are a popular method to bring about knowledge on the themes that are important for society and its sustainable development, for example, natural resources management, energy efficiency, urban planning, or climate change, among others [7][11][24][25][26].
Den Haan and Van der Voort [26] examined the current state of the art of different evaluation methods and procedures used to assess social learning outcomes of collaborative serious games by investigating 42 relevant publications and found that most evaluations focused on cognitive learning, few on normative and about half of the reviewed evaluations also on relational learning. In terms of tools, pre–post measures and qualitative interviews have been most widely used, mostly through self-reflective questions. Gameplay participant feedback is also widely used, as it is a relatively inexpensive and quick method for rapid assessment of the immediate impact of the game, and, indeed, is suitably robust when collated over a large number of game workshops (see in [2]). A multi-method evaluation approach is also recognised to fit well with serious gaming in general [26][27].
Some drawbacks related to the attempts to assess the effectiveness of serious games, especially in terms of behaviour change and social learning, are covered by several scholars [13][21][28], for example, the doubt of some researchers in the validity of the use of self-reflective questionnaires to assess the motivational aspects or that often no pretest of knowledge is implemented, which may unintentionally ignore the fact that differences in ultimate learning outcomes could actually be due to knowledge differences between individuals or groups at the start of the intervention [28]. Additionally, they discovered that players’ motivation as an important aspect in cases where the effectiveness of digital game-based learning is not always assessed or the type of motivation measured [13]. For assessing motivation, it is advised to assess the motivation to play the game, especially enjoyment, fun, and immersion; the motivation to continue using the game; and motivation towards learning the educational content of the game [29].
Researchers also discuss a variety of solutions for choosing more accurate evaluation approaches. In-game assessment becomes important to give information about the players’ or learners’ progress to the researcher and analyses the players’ competences at various levels during the play [30]. Indeed, the game itself can generate valuable assessment data by documenting how the player acts and chooses strategies inside the game [31][32]. Thus, serious games open up rich possibilities for data collection with the aim of understanding a wide range of human actions and behaviours [2][33]. Moreover, the need to augment survey-based evaluation scales with qualitative insights gained through observation and interaction with end users is highly recommended [4], thus emphasising the need for mixed-method approaches while evaluating the game design development and final outcomes.
Inevitably, using serious games can be beneficial in educating and raising awareness about bioeconomy topics, which is directly related to the focus of this research. Over the last couple of years, there have been several EU projects developed to raise people’s awareness about the bioeconomy and policies related to the topic. The EU-funded project BIOWAYS [34] has released two games—“Bio…What?” and “BIOChallenge”—where the players can discover how different raw materials can be used to produce everyday things and test their bioeconomy knowledge. The AllThings.bioPRO project, based on the outcomes of which this research is developed, has also created a bioeconomy game aiming to involve society in the transition to a more circular economy by introducing the key bioeconomy concepts and knowledge to the citizens and/or consumers. By following the overview of the widely applied evaluation methods and involving a mix of comprehensive evaluation tools, the case study provides learning opportunities in the discussions of serious game designs and potential added value by undertaking this in a fully participative way.

3. Co-Creation and Co-Design

The importance of user participation in designing innovative products and services has received growing recognition [35]. The concept of such open innovation promotes the idea that innovation can be enhanced by involving external stakeholders beyond the traditional boundaries of an organisation. “Co-creation”, “co-design” and “design thinking” are keywords that are often applied in open innovation approaches [35][36][37]. These concepts originally emerge from transformative processes in the entrepreneurial world by bringing together users and producers for a collaborative creation of new products and services. Fields like research and education have increasingly taken up this approach [38]. This shift from the traditional expert-driven approach emphasises the need to involve end users, such as learners, educators, domain experts, and even the general public in the ideation and design processes. One notion of co-creation is that if a user is involved in the production of a good or service, the end value will be enhanced because the user/customer can tailor the product as he or she would prefer [24][37][39][40].
Co-creation itself is a broad term used to denote stakeholder involvement in creating something, whether it be a specific object, outcome, product, or service [37][41][42]. There is no single universally accepted ‘official’ definition of co-creation, as the term is used in various disciplines and contexts [43]. However, several definitions offered, for example, by Sanders and Stappers [36], Unalab [44], and the SISCODE project [45], provide a comprehensive understanding. Morello et al. [42] define co-creation as a “collaborative approach to engagement which allows stakeholders to collectively design and build more inclusive and sustainable mechanisms for change” (p. 93), which has become a popular method of engagement in different areas of policy and decision-making.
Co-design (also known as participatory design) can be a part of co-creation and refers to involving stakeholders in designing something and combining collective creativity throughout the entire design procedure [36][46]. Usually, involved stakeholders are the potential users of the designed result (product, service, or otherwise) or other relevant and/or interested parties. Co-design recognises the expertise and perspectives of all participants, empowering them to contribute their knowledge and insights to create solutions that better meet their needs. The method’s flexibility to “accept multiple perspectives and work with a wide range of stakeholders” makes it suitable for application in various contexts [47] (p. 4). If used properly, it is a powerful tool that not only helps to ensure that the final result meets its users’ needs but also that there is a relationship between the designer and the users/stakeholders. Similar to co-creation, there is no single universally accepted ‘official’ definition of co-design, as the term is used in various disciplines and fields. However, there are a few recognised definitions that provide a comprehensive understanding of the concept, for example, those provided by Sanders and Stappers [36], Kleinsmann and Valkneburg [48], Steen [49], and the Design Council [50].
Concepts of co-creation and co-design intertwine closely with another popular strand: the design thinking approach. Design thinking is closely connected with interdisciplinarity. Its tools are helpful to visualise content, knowledge, relations, and mechanics and develop them further in iterative workshops with various stakeholders [11][51][52]. Design thinking offers a range of methods and tools that support collaboration and co-design, such as ideation techniques, empathy mapping, prototyping, and user testing [52][53]. Together, design thinking and co-creation, as well as co-design, foster an environment that encourages collective creativity, user participation, and the development of solutions that address real-world challenges.
Co-creation and co-design in any domain or innovation process should be viewed as a continuous process: neither can be a one-time event but should be thought of as a process in which stakeholders are engaged and followed up on. As such, long-term engagement requires what is commonly known as ‘community building’, i.e., recruiting participants to not only offer up their ideas but also to learn and collaborate with others (often cross-sectoral actors) while working towards a mutual goal. Some key steps to foster long-term stakeholder engagement and its orchestration are packaged by Unalab and other toolboxes [53][54]. They focus on community building, setting up processes and platforms for collaborative learning, guaranteeing transparency for all participants in the process, planning for options to integrate newcomers to the process at any stage and orchestrating and managing the system so that all components are integrated into their specific needs.

4. Benefits of Co-Creation and Co-Design

Since the notions of co-creation and co-design have become more widely accepted in different sectors and domains and methods of design thinking increasingly applied in innovation processes, the value and benefits of these new approaches have been under focus. Co-design brings about two overarching advantages, demonstrating its ability to generate original and valuable design concepts for users [55], while also enhancing overall product quality [56]. Researchers have found that a co-design approach facilitates a greater number of design ideas compared to non-co-design methods [57]. Moreover, innovations proposed by users during co-design activities exhibit a higher level of originality when compared to those generated solely by professional developers [58][59]. In service design contexts, co-design offers a range of benefits, including a deeper understanding of end user’s needs, the generation of more original and valuable ideas from users’ perspectives, and the creation of a better alignment between services and customer/user requirements [59]. Ultimately, co-creation and co-design enable enhanced service experiences and higher service quality [35].
Regarding the benefits of co-creation and codesign in developing serious games, some case study examples are promising [5][7][11][13][19][25][60]. For example, Lanezky et al. [11] analysed the development process of the serious board game “Changing the Game—Neighbourhood” and found that the influence of all participants of the co-design phases was crucial for the development process of the serious game, especially for the expansion of the learning content, the improvement in the gameplay, and the balancing of the difficulty level. All and Nunez Castellar’s study of the educational game for road safety [61] indicates that co-design can be a source of additional ideas on top of other research methods such as state-of-the-art analysis and expert consultation and thus can lead to more effective interactive content creation. In several other cases, including the target groups in the co-design process of a serious game helped create an understanding of current behaviour, preferences, and needs of the user, thereby enhancing the uptake and adherence of the intervention (serious game) [19][62][63].
However, co-creation and co-design do not usually deliver ready-to-use ideas or complete insight into the themes or game mechanics that would attract the whole target group. Rather, one should regard such a participatory design process as a useful source of inspiration for concept development and a method to broaden the perspective of game developers and researchers [12]. Therefore, it is useful to understand the qualitative value of the co-creation and co-design and find the best model, tools, and methods to foster the process to gain maximum benefits for all stakeholders—developers and participants. Kristensson et al. [37] focused on the conditions that lead to the benefits of co-creation in new technology-based services. Their empirical data suggest that user involvement during new product development should consider the following key strategies: (1) users identifying needs in their own setting of use; (2) users identifying needs in their various roles; (3) providing users with analytical tools; (4) motivating users via the apparent benefit to be gained from their involvement; (5) non-reliance on brainstorming when generating ideas; (6) involving users not having too much knowledge of technology; and (7) the involvement of a heterogeneous group of users to ensure that a diversity of ideas is provided for future services. Similar and connected principles that characterise a co-creative process and make it inviting for participants were phrased in 2017 by Jansen and Pieters [64]: (1) togetherness: there is an equal collaboration between all internal and/or external parties; (2) end users: they must play a central role in the overall process; (3) ongoing: the process is ongoing and participative in every phase; (4) productive: it leads to the implementation of the co-created solution; (5) transparent: relevant information is accessible to all; (6) supported: supported by all stakeholders; and (7) value-driven: results in value creation for end users and involved parties.

5. Evaluating Co-Creation and Co-Design

While there is widespread agreement on the value of involving end users in the design process of products or services they may eventually use, there is a need for concrete evidence regarding the impact of such involvement [35]. Thus, an important part of any participatory approach is its evaluation to understand the quality and impact of the process. This entails monitoring the process from inception to conclusion and gathering various types of data on the users engaged, activities conducted, and results achieved. Nevertheless, few studies have thoroughly examined the precise and multifaceted effects of end user involvement on the actual design outcomes being developed, especially in the game design field [65].
Evaluating co-creation and co-design is mainly accomplished through a mix of quantitative and qualitative research methods such as counting facts and figures, conducting interviews, and questionnaires [60] that provide data on users/participants, e.g., age, gender; how participants are engaging, how often, how long, and which activities; and feedback about participants’ experiences [66]. The user experience and satisfaction with the process is an important criterion for successful co-creation and co-design [43][45]. However, the non-linear nature of these processes can pose a challenge for evaluation, especially because co-design can include a wide range of activities that can make it difficult to directly link and attribute the co-creation and co-design with the outcomes; co-design may not even have predefined outcomes and different participants in co-design may engage in different ways or have different experiences. As such, monitoring the co-creation process and co-design throughout its various stages and constantly recording users’ behaviour, engagement patterns, and opinions can provide the main insights into assessing the impact of the co-creation and co-design approach. If available, quantitative data such as participant numbers is helpful as well in supporting evaluation, providing both additional information and a means of validating the qualitative input, but it alone provides an insufficient basis for evaluating the involvement of stakeholders [67][68]. It is therefore important to combine several methods and tools to gather input for evaluation, especially highlighting collective discussions between participants, using interviews and direct contact with evaluation target groups, combined with more quantifiable and standard surveys helping to keep track of progress over time, and collecting standardised information from the evaluation target groups.
By fostering effective co-creation and co-design in serious game development, the AllThings.bioPRO project aimed to ensure that the game meets the needs of the target audience, addresses complex issues and provides an engaging and impactful experience. In the following sections, the process is described in detail and lessons learned are discussed.

This entry is adapted from the peer-reviewed paper 10.3390/su151813364

References

  1. Hammady, R.; Arnab, S. Serious Gaming for Behaviour Change: A Systematic Review. Information 2022, 13, 142.
  2. Flood, S.; Cradock-Henry, N.A.; Blackett, P.; Edwards, P. Adaptive and Interactive Climate Futures: Systematic Review of ‘Serious Games’ for Engagement and Decision-Making. Environ. Res. Lett. 2018, 13, 063005.
  3. Stanitsas, M.; Kirytopoulos, K.; Vareilles, E. Facilitating Sustainability Transition through Serious Games: A Systematic Literature Review. J. Clean. Prod. 2019, 208, 924–936.
  4. Moizer, J.; Lean, J.; Dell’Aquila, E.; Walsh, P.; Keary, A.; O’Byrne, D.; Di Ferdinando, A.; Miglino, O.; Friedrich, R.; Asperges, R.; et al. An Approach to Evaluating the User Experience of Serious Games. Comput. Educ. 2019, 136, 141–151.
  5. Tuli, N.; Mantri, A. A Co-Design Methodology to Design and Develop Serious Educational Games: Integrating the Actors and Challenges in Each Phase; Springer: Berlin/Heidelberg, Germany, 2018; p. 52. ISBN 9789811307546.
  6. Susi, T.; Johannesson, M.; Backlund, P. Serious Games—An Overview; School of Humanities and Informatics, University of Skövde: Skövde, Sweden, 2007.
  7. Gugerell, K.; Zuidema, C. Gaming for the Energy Transition. Experimenting and Learning in Co-Designing a Serious Game Prototype. J. Clean. Prod. 2017, 169, 105–116.
  8. Raphael, C.; Bachen, C.; Lynn, K.-M.; Baldwin-Philippi, J.; McKee, K.A. Games for Civic Learning: A Conceptual Framework and Agenda for Research and Design. Games Cult. 2010, 5, 199–235.
  9. All, A.; Looy, J.; Nunez Castellar, E. Co-Designing Interactive Content: Developing a Traffic Safety Game Concept for Adolescents. In Proceedings of the 6th European Conference on Games Based Learning (ECGBL-2012), Cork, Ireland, 4–5 October 2012; p. 20, ISBN 978-1-908272-70-6.
  10. Sutcliffe, M. Simulations, Games and Role-Play. In The Handbook for Economic Lecturers; The Higher Education Academy Education Network: Bristol, UK; pp. 1–26.
  11. Lanezki, M.; Siemer, C.; Wehkamp, S. “Changing the Game—Neighbourhood”: An Energy Transition Board Game, Developed in a Co-Design Process: A Case Study. Sustainability 2020, 12, 10509.
  12. All, A.; Nunez Castellar, E.; Looy, J. A Systematic Literature Review of Methodology Used to Measure Effectiveness in Digital Game-Based Learning. In Proceedings of the 7th European Conference on Game Based Learning, Paisley, UK, 25–26 October 2007; Publishing International Limited: Gloucestershire, UK, 2013; Volume 2, p. 616.
  13. All, A.; Nunez Castellar, E.; Castellar, N.; Looy, J. Measuring Effectiveness in Digital Game-Based Learning: A Methodological Review. Int. J. Serious Games 2014, 1, 2.
  14. Moreno Ger, P.; Burgos, D.; Martinez-Ortiz, I.; Sierra, J.; Fernández-Manjón, B. Educational Game Design for Online Education. Comput. Hum. Behav. 2008, 24, 2530–2540.
  15. Pagulayan, R.; Keeker, K.; Wixon, D.; Romero, R.; Fuller, T. User-Centered Design in Games. Hum. Comput. Interact. Handb. 2012, 20126252, 795–822.
  16. Ávila-Pesántez, D.; Rivera, L.A.; Alban, M.S. Approaches for Serious Game Design: A Systematic Literature Review. Comput. Educ. J. 2017, 8, 3.
  17. Gameil, A.A.; Al-Abdullatif, A.M. Using Digital Learning Platforms to Enhance the Instructional Design Competencies and Learning Engagement of Preservice Teachers. Educ. Sci. 2023, 13, 334.
  18. Homer, B.D.; Hayward, E.O.; Frye, J.; Plass, J.L. Gender and Player Characteristics in Video Game Play of Preadolescents. Comput. Hum. Behav. 2012, 28, 1782–1789.
  19. Dekker, M.R.; Williams, A.D. The Use of User-Centered Participatory Design in Serious Games for Anxiety and Depression. Games Health J. 2017, 6, 327–333.
  20. Menestrina, Z.; Pasqualotto, A.; Siesser, A.; Venuti, P.; De Angeli, A. Engaging Children in Story Co-Creation for Effective Serious Games. Sustainability 2021, 13, 10334.
  21. Wainess, R.; Oneil, H.; Baker, E. Classification of Learning Outcomes: Evidence from the Computer Games Literature. Curric. J. 2005, 16, 455–474.
  22. Wu, X.; Liu, S.; Shukla, A. Serious Games as an Engaging Medium on Building Energy Consumption: A Review of Trends, Categories and Approaches. Sustainability 2020, 12, 8508.
  23. Tan, C.; Nurul-Asna, H. Serious Games for Environmental Education. Integr. Conserv. 2023, 2, 19–42.
  24. Ampatzidou, C.; Gugerell, K. Participatory Game Prototyping—Balancing Domain Content and Playability in a Serious Game Design for the Energy Transition. CoDesign 2019, 15, 345–360.
  25. Billger, M.; Kain, J.-H.; Niwagaba, C.B.; McConville, J.R. Lessons from Co-Designing a Resource-Recovery Game for Collaborative Urban Sanitation Planning. IOP Conf. Ser. Earth Environ. Sci. 2020, 588, 042041.
  26. Den Haan, R.-J.; Van der Voort, M.C. On Evaluating Social Learning Outcomes of Serious Games to Collaboratively Address Sustainability Problems: A Literature Review. Sustainability 2018, 10, 4529.
  27. Wouters, P.J.M.; Spek, E.; Oostendorp, H. Current Practices in Serious Game Research: A Review from a Learning Outcomes Perspective. Games-Based Learn. Adv. Multi-Sens. Hum. Comput. Interfaces Tech. Eff. Pract. 2009, 232–250.
  28. Clark, R. Learning from Serious Games? Arguments, Evidence, and Research Suggestions. Educ. Technol. 2007, 47, 56–59.
  29. Schonau-Fog, H.; Bjorner, T. “Sure, I Would Like to Continue”: A Method for Mapping the Experience of Engagement in Video Games. Bull. Sci. Technol. Soc. 2012, 32, 405–412.
  30. Shute, V.; Rieber, L.; Van Eck, R. Games and Learning. Trends Issues Instr. Des. Technol. 2011, 321–332.
  31. Vincent, K.; Dougill, A.J.; Dixon, J.L.; Stringer, L.C.; Cull, T. Identifying Climate Services Needs for National Planning: Insights from Malawi. Clim. Policy 2017, 17, 189–202.
  32. de Suarez, J.; Suarez, P.; Bachofen, C.; Bachofen, C. Games for a New Climate: Experiencing the Complexity of Future Risks; Boston University: Boston, MA, USA, 2012.
  33. Juhola, S.; Driscoll, P.; Mendler de Suarez, J.; Suarez, P. Social Strategy Games in Communicating Trade-Offs between Mitigation and Adaptation in Cities. Urban Clim. 2013, 4, 102–116.
  34. EU Horizon 2020 Project BIOWAYS. “Serious Games”. Available online: https://www.bioways.eu/bio-learn/serious-games (accessed on 13 July 2023).
  35. Cockbill, S.A.; May, A.; Mitchell, V. The Assessment of Meaningful Outcomes from Co-Design: A Case Study from the Energy Sector. She Ji J. Des. Econ. Innov. 2019, 5, 188–208.
  36. Sanders, E.B.-N.; Stappers, P.J. Co-Creation and the New Landscapes of Design. CoDesign 2008, 4, 5–18.
  37. Kristensson, P.; Matthing, J.; Johansson, N. Key Strategies for the Successful Involvement of Customers in the Co-creation of New Technology-based Services. Int. J. Serv. Ind. Manag. 2008, 19, 474–491.
  38. Waltenberg, T.; Steinhaus, N. D3.1: Report of the First Sequence of Focus Groups and Co-Creation Workshops. Deliverable 3.1, AllThings.BioPRO, H2020 Grant No. 887070; Wissenschaftsladen Bonn e.V.: Bonn, Germany, 2021.
  39. Ampatzidou, C.; Gugerell, K. Mapping Game Mechanics for Learning in a Serious Game for the Energy Transition. Int. J. E-Plan. Res. 2019, 8, 1–23.
  40. Burby, R.J. Making Plans That Matter: Citizen Involvement and Government Action. J. Am. Plann. Assoc. 2003, 69, 33–49.
  41. Lusch, R.F.; Vargo, S.L.; O’Brien, M. Competing through Service: Insights from Service-Dominant Logic. J. Retail. 2007, 83, 5–18.
  42. Morello, E.; Mahmoud, I.; Gulyurtlu, S.; Boelman, S.; Davis, H. CLEVER Cities Guidance on Co-Creating Nature-Based Solutions: PART I—Defining the Co-Creation Framework and Stakeholder Engagement. Deliverable 1.1.5, CLEVER Cities, H2020 Grant No. 776604; Politecnico di Milano: Milan, Italy, 2018.
  43. Meister Broekema, P.; Horlings, L.G.; Bulder, E.A.M. Understanding the Value of Co-Creation for Social Innovation Interpretations of Social Innovation and Co-Creation in European Policy-Related Documents between 1995 and 2018. Innov. Eur. J. Soc. Sci. Res. 2022, 35, 291–308.
  44. Co-Creation in UNaLab|UNaLab. Available online: https://unalab.eu/en/co-creation (accessed on 13 July 2023).
  45. A SISCODE Definition of Co-Creation|Ecsite. Available online: https://www.ecsite.eu/activities-and-services/news-and-publications/siscode-definition-co-creation (accessed on 13 July 2023).
  46. De Jans, S.; Van Geit, K.; Cauberghe, V.; Hudders, L.; De Veirman, M. Using Games to Raise Awareness: How to Co-Design Serious Mini-Games? Comput. Educ. 2017, 110, 77–87.
  47. Manohar, A.; Smith, M.; Calvo, M. Capturing the “How”: Showing the Value of Co-Design through Creative Evaluation; Brunel University London: London, UK, 2016.
  48. Kleinsmann, M.; Valkenburg, R. Barriers and Enablers for Creating Shared Understanding in Co-Design Projects. Des. Stud. 2008, 29, 369–386.
  49. Steen, M. Co-Design as a Process of Joint Inquiry and Imagination. Des. Issues 2013, 29, 16–28.
  50. Design Council Design Council Website. Available online: https://www.designcouncil.org.uk/ (accessed on 13 July 2023).
  51. Robertson, T.; Simonsen, J. Challenges and Opportunities in Contemporary Participatory Design. Des. Issues 2012, 28, 3–9.
  52. Plattner, H.; Meinel, C.; Leifer, L. Design Thinking Research: Studying Co-Creation in Practice; Springer: Berlin/Heidelberg, Germany, 2012; p. 277. ISBN 9783642216428.
  53. UNALAB Toolkit. Available online: https://unalab.enoll.org (accessed on 13 July 2023).
  54. Jeffery, N. Stakeholder Engagement: A Roadmap to Meaningful Engagement; Doughty Centre, Cranfield School of Management: Wharley End, UK, 2009.
  55. Kristensson, P.; Magnusson, P.R. Tuning Users’ Innovativeness During Ideation. Creat. Innov. Manag. 2010, 19, 147–159.
  56. Roser, T.; Samson, A.; Humphreys, P.; Cruz-Valdivieso, E. Co-Creation: New Pathways to Value: An Overview. Promise LSE Enterp. 2009.
  57. Mitchell, V.; Ross, T.; May, A.; Sims, R.; Parker, C. Empirical Investigation of the Impact of Using Co-Design Methods When Generating Proposals for Sustainable Travel Solutions. CoDesign 2016, 12, 205–220.
  58. Magnusson, P.R. Benefits of Involving Users in Service Innovation. Eur. J. Innov. Manag. 2003, 6, 228–238.
  59. Steen, M.; Manschot, M.; Koning, N. Benefits of Co-Design in Service Design Projects. Int. J. Des. 2011, 5, 2.
  60. Beg, I.; Looy, J.; All, A. Evaluation of the Game Development Process of a Location-Based Mobile Game; Academic Conferences International Limited: Reading, UK, 2014; Volume 1.
  61. All, A.; Nunez Castellar, E. An Evaluation of the Added Value of Co-Design in the Development of an Educational Game for Road Safety. Int. J. Game-Based Learn. 2013, 3, 1–17.
  62. Fleming, T.; Dixon, R.; Frampton, C.; Merry, S. A Pragmatic Randomized Controlled Trial of Computerized CBT (SPARX) for Symptoms of Depression among Adolescents Excluded from Mainstream Education. Behav. Cogn. Psychother. 2012, 40, 529–541.
  63. Roepke, A.M.; Jaffee, S.R.; Riffle, O.M.; McGonigal, J.; Broome, R.; Maxwell, B. Randomized Controlled Trial of SuperBetter, a Smartphone-Based/Internet-Based Self-Help Tool to Reduce Depressive Symptoms. Games Health J. 2015, 4, 235–246.
  64. Jansen, S.; Pieters, M. The 7 Principles of Complete Co-Creation; Laurence King Publishing: London, UK, 2018; ISBN 978-90-6369-473-9.
  65. Visser, F.S.; Stappers, P.J.; van der Lugt, R.; Sanders, E.B.-N. Contextmapping: Experiences from Practice. CoDesign 2005, 1, 119–149.
  66. Abrams, M.M.; McLeod, R. Implementing and Evaluating Co-Design. A Step-By-Step Toolkit; New Philanthropy Capital: London, UK, 2019.
  67. Riedel, V.; Dziekan, K.; Müller, S.; Abraham, M.; Kettner, S.; Daubitz, S. Evaluation Matters. A Practitioners’ Guide to Sound Evaluation for Urban Mobility Measures; Waxmann: Münster, Germany, 2013; ISBN 978-3-8309-2881-2.
  68. Asser, A.; Tatar, M.; Henahan, R. D7.2: Evaluation of Stakeholder Involvement Process. Deliverable 7.2, AllThings.BioPRO, H2020 Grant No. 887070; Institute of Baltic Studies: Tartu, Estonia, 2022.
More
© Text is available under the terms and conditions of the Creative Commons Attribution (CC BY) license; additional terms may apply. By using this site, you agree to the Terms and Conditions and Privacy Policy.
This entry is offline, you can click here to edit this entry!
Video Production Service
Feedback