1. Introduction

Digitalization has the potential to have a drastic impact on all steps of the energy value chain as well as on all stakeholders, including the environment, the overall society, and the national economy. Good knowledge of the functionality of digital applications and their impact on all stakeholders is necessary. On the one hand, it enables a broad societal and political discussion to set the general targets and guidelines for the digitalization process and on the other hand, it is an important piece of information for companies to develop and sustainably implement digital applications.

Therefore, a potential methodology to holistically assess and evaluate digital applications in the electricity sector is suggested. While the methodology is outlined in this entry, a follow-up article will present a detailed description, an applied use case of the methodology and a critical reflection of the approach.

2. Suggested Evaluation Methodology for Digital Applications

A variety of conventional as well as rather recent methods, such as living labs and design thinking, are already applied in the development of new products, services, and business models. However, a holistic analysis of benefits and potential risks of digital applications, taking into account all impacted stakeholders, provides an effective way to identify and solve potential impediments and drawbacks early on.

The following suggested methodology is presented as an outlook in ^[1]. It offers a basic framework to perform a holistic analysis of technical, ecological, social and economic aspects of digital applications. A combination of a multi-criteria analysis (MCA), a life cycle assessment (LCA), and semi-structured interviews is suggested. In the following figure, the overall concept and basic functionality of the combination of methodologies is depicted. The concept can be adapted to the diverse kinds of digital applications and different analysis depths, and it allows the inclusion of quantitative as well as qualitative criteria. The result is an overall evaluation with reduced complexity including specific highlights, risks and respective solution options.

As depicted in the above figure, the basic structure for the evaluation is a multi-criteria analysis. However, the assessment of the different criteria defined in the MCA is conducted either within the life cycle assessment or via the semi-structured expert interviews. This offers the flexibility to either perform a more qualitative “Quick Check” based mostly on expert interviews (or even literature review) or if required a qualitative “Deep Analysis” via the LCA.

Multi-criteria analysis (MCA)

The MCA is a tool to assess different alternatives with high complexity, high uncertainty, inhomogeneous criteria, inhomogeneous availability of information and multiple perspectives / stakeholders. Diverse assessment criteria can be evaluated on a common basis, and a compensation between criteria is possible. Therefore, the complexity of the result can be reduced, which is a required prerequisite for broader social or political discussions. Furthermore, weighting factors can be used to reflect different (stakeholder) perspectives.

A general description of the MCA methodology is given for example in ^[2][3][4][5]. The application of the MCA in the energy sector and for sustainability assessment is described in ^{[6][7][8][9][10]} and ^[11].

Life cycle assessment (LCA)

The life cycle assessment is mainly a tool to assess the ecological (and social) sustainability of products and services along the entire life cycle, i.e. cradle-to-grave. Furthermore, it can help to identify risks and bottlenecks and provide a basis to identify potential solutions. The cradle-to-grave approach, including the supply of primary materials, production, transport, use-phase and recycling and/or landfilling ensures the integration of factors which are not reflected in purely economically driven methods and thereby avoids that the related potential impacts and costs are missed.

In the context of the proposed combination of methods the LCA is used to assess mainly technological and ecological but also economic and social aspects which are mostly quantifiable and are first-order effects (direct effect e.g. use of resources for hardware) and directly attributable effects of second-order (e.g. direct reduction of energy consumption due use of application).

The LCA is defined by  ^[12] and ^[13].  Literature on the general application of LCA is for example given by ^{[14][15][16][17][18][19]}. The application of LCA in the energy sector is described for example in ^{[20][21][22][23]}. How the LCA can be used to assess social aspects is described in ^[24].

Semi-structured Interviews

While the LCA is used to assess mostly quantifiable first-order and, to a certain extent, second-order effects, the semi-structured interviews are used when information availability or reliability is poor. Therefore, interviews are mostly applicable for third-order effects, e.g. rebound effects as well as some second-order effects, which cannot be attributed easily to the digital application under assessment, e.g. effects, which occur in the larger energy system and are also influenced by a variety of other factors. In the interview, the identified criteria are evaluated based on the logic defined in the MCA. The methodological background of expert interviews is described in ^[25].

A first test of the described methodology is currently being conducted for the German “smart meter” roll-out in cooperation with “smart meter” manufacturers. While an overall evaluation result cannot yet be given, first risks/bottlenecks have already been identified. One risk is the use of conflict (e.g., tin, tantalum) and scarce/critical (e.g., rare earth elements) materials. However, this risk is not specific to “smart meters” but applies to most modern communication devices. Besides that, some manufacturers of “smart meters” have managed to minimize or even eliminate the use of certain conflict and scarce materials. Another risk is a potential rebound effect. Although field tests have shown a reduction in energy consumption after the implementation of “smart meters”, it is unknown to what extent these reductions are permanent. A lower reduction of energy demand could offset the life cycle green house gas (GHG) balance. This risk could be met in the future with automated demand response mechanisms for “smart” household devices based on, for example, price signals. Furthermore, missing standards such as for “smart home” network communication protocols and interfaces could lower the implementation speed of applications and thereby reduce overall benefits.

Full application of the methodology and discussion of the results, including an evaluation of the suitability of the methodology is yet to come. However, the first experiences of the test suggest that the proposed methodology as such could be suitable for the purpose of a holistic evaluation of digital applications.

3. Conclusions

Most digital applications do not only cause benefits but also have risks of downsides. Therefore, a good understanding of both benefits and risks from all stakeholders’ perspectives at an early stage of development is essential to find solutions to mitigate the risks and make full use of the benefits. A methodology for a holistic assessment and evaluation of digital applications in the energy sector is presented. The methodology consists of a combination of multi-criteria analysis (MCA), life cycle assessment (LCA), and semi-structured expert interviews. Going forward the suggested methodology needs to be further detailed, tested and revised. It could also be adapted to be suitable for digital applications in other, non-energy sectors. Overall, the aim is to create an evaluation basis that provides a structured approach to holistically assess digital applications and provide assessment results with reduced complexity.

The publication can be found here: https://www.mdpi.com/2076-3417/9/24/5350/htm