Evaluating Public Services Considering Expectations of Users

Evaluating Public Services Considering Expectations of Users: History

Please note this is an old version of this entry, which may differ significantly from the current revision.

Subjects: Public Administration

Contributor:

Evaluating public services has become an important task in order to direct actions that may positively affect with the quality of the service provided by governments. To undertake an effective evaluation, it is necessary to analyze data and information on the impact of the services. A wide range of studies have been proposed to measure how an organization delivers its services according to the expectations of the stakeholders. Public sector services should be designed to allow citizens to manage their lives most efficiently. Such designs should be based on an understanding of the experience and the services shaped to integrate into the user’s world. A good strategy will avoid (or mitigate) failure, especially of electronic government projects.

digital transformation
service quality
public services evaluation

1. Introduction

The relationship between governments and users has changed in recent decades because of digital transformation. The use of technologies not only improves the performance of the service but also increases the service’s reach [1]. The need to assess the effectiveness of public services arises as governments increase the development of online systems to deliver public services [2]. Such evaluation efforts can enable government organizations to determine if they are capable of doing the task required and delivering services as expected. In this context, performance based models emerge to evaluate the effectiveness and efficiency of public services, in order to make them closer and more acceptable to users [3,4].

A great challenge to evaluate the performance of public services is to obtain valid data to guide the Public Administration with direct actions that may positively interfere in the quality of the service. The first models for service evaluation used to consider organizational contexts and dimensions that could impact the quality of a service, such as empathy and courtesy [5,6,7]. With technological evolution, the perception of the user about a service changed. The previous evaluation models also evolved and started to consider new dimensions, such as usability [8,9].

Evaluating public services from the perspective of the user became an opportunity to detect which factors hinder or facilitate their impact on society. However, there is no single unitary indicator in the literature to assess the effectiveness of public services [10]. For this reason, a mapping study of the main differences between evaluation-based models and their applications would be an excellent guide for the Public Administration to implement an effective evaluating model for its services.

2. Evaluation Approaches

Our study identified four evaluation approaches of public services from the perspective of users: quality of services, user satisfaction, user experience, and success and acceptance of Information Systems (IS).

2.1. Service Quality

Quality is a broad and complex topic. It covers everything from institutional practices to the application of specific statistical tools and techniques to improve and monitor quality levels [18]. According to [45], the quality of services is intangible and therefore its measurement is complex. In general, the evaluation models based on the quality of services seek to measure the perceptions of the users against the results of service delivery.

Most studies that take this approach use or are based on the SERVQUAL [6] model, which adopts a scale divided into five dimensions: tangibles, responsiveness, reliability, assurance, and empathy. This instrument assesses users’ expectations and perceptions of a service. Based on this, several studies focus on the perceptions of quality, which is often compared to the expected quality to obtain a final metric. Thus, the user’s expectation and perception of the quality of the service received are compared.

Furthermore, some of these studies state that user satisfaction is a direct result of the quality of services. It is frequently pointed out that, despite the focus on the perception of users, a complete evaluation of the quality must include the entire service delivery mechanism, analyzing, for example, its technical and functional aspects.

According to [18], the most important factors that influence public service quality are:

Access to the service;
Communication level;
An understandable administrative system;
The ability to provide a flexible and quick reply;
Receptivity to service;
Politeness and kindness of the administration staff;
Credibility in the service supply;
Reliability and responsibility;
Security and quality of tangible aspects.

In [33], factors influencing the e-service quality of government portals were identified from an extensive review of research performed by academic scholars and practitioners. Seven constructs were identified from the analyses: citizen centricity, transaction transparency, technical adequacy, usability, complete information, privacy and security, and usefulness of information. These constructs can be used to assess the service quality of government portals.

2.2. User Satisfaction

User satisfaction is a psychological aspect related to the customer’s goals and needs. It serves as a benchmark tool for assessing user-oriented government strategies [37]. According to [24], satisfaction is a central field of interest for contemporary organizations. Other common dimensions to this approach are quality of services, customization or flexibility, utility, and user loyalty.

Evaluation based on user satisfaction generally comprises variables of service quality. Broadly speaking, service quality and use determine satisfaction. In [44], the authors argue that user satisfaction guarantees an immediate and significant response to the preferences and expectations of service users. However, the same authors also point out that many different models can be adopted to measure user satisfaction.

According to [36], governments ensure user satisfaction by utilizing the information and communication technology properly, specially the internet. By improving this channel of communication, the public sector ensures the accessibility and completeness of government information and service delivery in a more convenient way. However, the quality of websites is frequently overlooked during the design and implementation stages of online public services. Besides transparency, ease of navigation and comprehensive information require adequate monitoring resources, including targets for responses and reports. In fact, online users expects the organization to respond to their inquiries without delay. An immediate response will assist government website users to make decisions if it answers questions and resolves problems [29].

User satisfaction is a performance measurement that can be assessed with various techniques, for example Technology Acceptance Model (TAM), McLean and Delone Model, SERVQUAL, and EGOVSAT Model. It provides immediate, meaningful and objective feedback about user’s preference and expectations. On the other hand, the performance of public services will be evaluated in relation to a set of dimensions that indicate the strong and the weak factors affecting user satisfaction with public services [44].

2.3. User Experience

User experience encompasses all the tools, people, and processes involved in building a co-produced user experience [20]. It comprises all activities required of the user to obtain the service and/or fulfill their obligations to the government.

This approach correlates with the area of human–computer Interaction. As [38] suggest, it is closely linked to usability and can serve as an extension of it. Common dimensions to this approach are safety, flexibility or convenience, errors, and attractiveness.

According to [38], users have differing backgrounds, experiences and cultures. Users require effective, easy, and enjoyable interaction, which is key to successful use and acceptance of applications. In [23], a broad based research instrument was developed to capture perceptions of the user experience with the Nigeria Immigration Service website. They identified eight factors related to user experience: security and support, content and information, ease of use, benefits, barriers, convenience, trust, and website quality.

Usually, citizens do not have a clear understanding of the various functions of the service and how to navigate them, nor should they. In fact, [20] demonstrated that service usability can be improved by taking different perspectives of the service experience. However, the political will to make changes across government is missing. Public services will remain fragmented, reliant and difficult to consume until a conscious decision is made to evaluate the success of government based on the user experiences.

2.4. Success and Acceptance of IS

Understanding success in Information Systems (IS) is a complex challenge made more difficult when set in the public sector environment, while private sector studies may focus on profitability efficiency, quality, and reliability, public sector evaluation must combine these concerns with accountability, citizen trust, and the creation of public value [22].

To evaluate service delivery strategies, most of the studies identified in this SLR examine the success and acceptance of the IS based on the DeLone and McLean Model of Information Systems Success (D&M model) [21] and the Technology Acceptance Model (TAM) [47]. In [34], the authors emphasized that the DeLone and McLean IS success models are among few such theories and models that have helped researchers to establish some of the salient factors that influence the acceptance and use of public services by citizens before testing and post adoption. TAM [47] is a basic model used for technology acceptance studies. This model identifies the causal relationships between system design features, perceived usefulness, perceived ease of use, attitudes towards using, and actual usage behavior. It suggests that only ease of use and usefulness are insufficient to predict the behavior of users, as it is also necessary to include features such as system flexibility, system availability, information reliability, etc.

Studies that use acceptance of IS as an evaluation approach seek to measure service success by relating variables concerned with the overall quality, the constructs directly related to quality, and the behavioral and intentional variables of system use. The net benefit received by the user is also often analyzed. The variable of use—in its behavioral aspect—allows for determining the acceptance of the system. Then, the cyclical interrelationship between use, satisfaction and benefits makes it possible to point out the success of a system.

In conclusion, acceptance of the system and its subsequent use are vital points for the success and improvement of a user-oriented service.

3. Performance Evaluation of Public Services

Performance is a term that defines the successful fulfillment of one or more activities performed by an organization. In this sense, evaluating the performance of public services is an utmost task to reveal the full value of the service projects. In the public sector, performance is difficult to identify because the processes are more difficult to quantify than in the private sector where the main purpose is profit. This evaluation is also not effective without considering the perspectives of all the key issues held by each stakeholder, especially the final users [48].

The success of governmental organizations depends heavily the quality of the services they provide to citizens. People place strong importance on the attention the administration pays to the complaints they make. So, a useful and suitable reply system to deal with these complaints is one of the necessary improvements for all the administrations. As a consequence, government service managers should be able to reduce risks (e.g., investing valuable resources in service quality characteristics that may not work effectively).

However, it is important to know what dimensions of quality of public services contribute to enhance users’ satisfaction. As governments increase the development of online systems, it is necessary to assess the effectiveness of their digital services, while information systems (IS) success models have received much academic attention, little research has been conducted to assess the success of digital public services [21]. In this sense, there is a need to investigate whether the success of traditional IS models can be extended to online systems. Such evaluation efforts can enable government agencies to ascertain whether they can do the task required and delivering services as expected.

Currently, the user satisfaction dimension is strongly related with the management, use, and appearance of the online services, especially in town councils with a large population [25]. However, new technologies constitute a basic service to any citizen, regardless of the size of the town, so public actions are necessary to promote the use of new technologies. This would have a positive effect on the relationship between citizens and the public administration.

Service quality is another dimension related with the performance evaluation of public services. The quality of a service can help the public administration to make decisions that increase citizen satisfaction. Prior to building a service quality management system, the problems of the public service should be identified and the measurement tool of public service should be defined [18].

Finally, a limitation of most studies observed by our SLR is that the practical investigations of public services are based on a mandatory information system in a particular country and region, with specific characteristics. This can affect the generalizability of the results. Thus, the interpretation of the results should be confined to the countries and regions with similar settings.

4. Practical Utilization of Service Evaluation

The use of performance indicators by public sector organizations has been on the public sector management agenda in many governments around the world. This SLR identified works carried out to explore the evaluation of several applications according to the user’s perspective.

In [17], the authors provide a set of clear and useful e-health evaluation criteria that help improve the use of services by citizens, to integrate the e-health evaluation framework, and to address areas that require further attention in the development of future e-health initiatives. Other works, such as [2,36], focus on the assessment of services related to tax payment. The results point to strong connections between the following constructs: system quality, service quality, information quality, user satisfaction, and perceived usefulness.

Many works use indicators to evaluate the performance of local governments [18,25,37]. Local governments are administrative services that are smaller than a state. The term is used to contrast services at the nation-state level, which mention the central government or (where appropriate) the federal government. It is important to highlight the importance and shortage of studies about public service delivery and quality by local governments. Local organizations have the advantage of knowing better what their citizens want. This is very significant because the greater quality attributed to the public services by citizens, the less they will oppose financing them. This results in a better cooperation between citizens, and there will be a better match between what the citizens want and what the administration provides.

Finally, the need to know whether a public organization has met its aims has led to various forms of assessment. In this sense, it is necessary to develop more understandable forms to approach users more effectively. Some people find it difficult to understand forms they need to fill in. The use of plain language by the public administration is highly appreciated. In this sense, diverse means of communications (besides fax, telephone, Internet, etc.) can become valuable sources for users to evaluate their experiences with the service received. E-mail, comment sections, chat rooms, search features, broadcasting of government events, and web site personalization should facilitate connections and interaction between government institutions and users.

This entry is adapted from the peer-reviewed paper 10.3390/info13040162

© Text is available under the terms and conditions of the Creative Commons Attribution (CC BY) license; additional terms may apply. By using this site, you agree to the Terms and Conditions and Privacy Policy.