1. Introduction

In an era of digital transformations, traditional bureaucratic organizational structures tend to be replaced by e-government projects. Information and Communications Technology (ICT) has enabled a reduction in administrative costs and leveraged the immense potential of technology for more efficient public services. Digital public services (also called e-government) are delivered to the citizens by the government using ICTs. Examples of digital public services include income tax declarations, notification and assessment; birth and marriage certificate; renewing a driver’s license as well as other kinds of request and delivery of permissions and licenses. By using digital services, the government can deliver information and services to citizens anytime, anywhere, and on any platform or device.

The adoption of digital services in the public sector is complex and challenging, particularly in developing countries ^[1]. This is due to the inefficiency of public organizations, the shortage of skilled human resources, the poor ICT infrastructure, the low standards of living and the large rural population ^[2]. Many developing countries have implemented specific e-government initiatives for making a full use of the potential benefits of e-government ^[3]. In this sense, the demand for a systematic, continuous and effective assessment of the provision of digital public services contrasts with the lack of clarity regarding performance indicators as service evaluation is a broad theme ^[4].

The development of e-government creates a need for its continuous evaluation around the world ^[5]. There are several approaches to evaluate the performance of e-government in the literature. The readiness assessment, for example, examines the maturity of the e-government environment by evaluating the awareness, willingness, and preparedness of e-government stakeholders and identifying the enabling factors for the development of e-government ^[6]. The main advantage of this approach is using quantifiable indicators that provided an overview of the maturity of e-government ^[3]. However, using the readiness perspective to evaluate e-government is often criticized for neglecting the demands of citizens and the impact of digital services on citizens and the society.

2. Methods of Digital Public Services Evaluation

There have been many others attempts for evaluating e-government across the global regions. In the UK, the performance of digital public services is assessed considering three public value evaluation dimensions: quality of public service delivery, outcomes, and trust ^[7]. Quality of public service delivery is evaluated through the level of information provision, level of use, availability of choice, user satisfaction, user priorities, fairness, and cost savings. The European e-Government Action Plan 2016–2020 ^[8] assesses e-government as the reduction of administrative burden on citizens, improvement of citizens’ satisfaction and inclusiveness of public services. Russian Federation ^[9] assesses effectiveness as the quality of public services, trust, and outcomes. The Agency for the Development of Electronic Administration in France ^[10] proposes a framework for evaluating the public value of information technology in government with a focus on the financial benefits of e-government projects for citizens. One of the critical issues is how to evaluate and assess the successfulness of such projects. The traditional value assessment methods existing in the business field are not good enough to cope with the issue, as business and government hold different value perspectives and have different concerns ^[11].

Some of the more traditional models for service evaluation are based on efficiency, efficacy and effectiveness ^[12]. Effectiveness can be defined as the perception of changes made; efficacy refers to the extent to which the intended goals were achieved, and efficiency means doing more with fewer resources ^[13]. Among these three criteria, effectiveness plays a key role to produce desired social outcomes. Indeed, it aims to guarantee practical results, as it would be useless to have the most satisfactory result if it were not possible to realize the impact of the service on society. In this sense, effectiveness has a lot to do with the search for users’ satisfaction.

The great challenge for the measurement of effectiveness in the public sector is to obtain valid data to inform the Public Administration of the results the services. There is no consensus in the literature on what is the best indicator to measure the effectiveness of a public service. There is a gap on this subject, mainly regarding the importance of the indicators considering the needs of the user. In this sense, evaluating the effectiveness of public services from the perspective of the user is an opportunity to detect the factors that hinder or facilitate their impact on society.

3. Conclusions

The evaluation of public services has become an important aspect of the decision-making process by managers and public institutions. This evaluation increases the probability of obtaining better results and finding unexpected results. Monitoring and evaluation are always based on indicators that assist in decision-making, allowing for better performance, for more rational planning and for a clearer and more objective accountability.

The ‘useful’ indicator may also refer to the value that citizens attribute to their experience in public services and that can be understood as “public value”. In other words, it refers to the provision of services that are actually necessary and will be used ^[14]. It provides a new way of thinking about the evaluation of government activities and a new conceptualization of the public interest and the creation of social value. In a citizen-centered approach, developing services without considering the demands of users may lead to low rates of service use. In this sense, the Public Administration should minimize wasteful and unnecessary public services to save costs that may generate fiscal stress. Providing appropriate services narrow the state management apparatus according to its core functions, whereby it can provide better services and respond to demands for transparency and accountability.

Due to the nature of public services, effectiveness can be considered based not only on the quality perceived individually by users, but also on its social interest. The social interest of a public service is related to the government’s duty-power in guaranteeing the basic rights of the citizens ^[15]. For essential services—for example, those related to health—the usefulness and social need for the service are clear to the citizens. On the other hand, the social importance of other services—for example, the payment of fees and taxes—is sometimes not so clear as the direct benefits are not noticeable for the citizen. This observation is important since the application and implication of service evaluation may be contingent on the perspective taken ^[16].

The literature is still unclear regarding how to measure the personal and social impact of public services. This is a gap for advances in research in this area, especially in developing countries ^[17]. Such impact could be analyzed, for example, considering the category of service delivery, i.e., the ways in which services are delivered to users. In this way, it is possible to capture a clear articulation of the nature, boundaries, components and elements of specific e-service experiences, and to further investigate the interaction between these factors and the dimensions of service quality ^[18]. Another possibility is to perform ethnography studies on the digital transformations of services. Ethnographic may be used as method to collect qualitative information from the observation of people carrying out daily tasks that interact in complex social environments ^[19]. An ethnographic analysis is pertinent when new technologies are studied as it helps to find out and explain why many services are not welcome or used.

Social indicators end up being less frequently assessed in the literature on the evaluation of digital services, but that does not make them less relevant. In recent years, the concept of public value has become popular in the United States, the European Union, Australia and even in developing countries due to its ability to investigate the performance of public services from the point of view of citizens ^[20].