Evaluation Criteria for Tools Supporting Remote Work: History
Please note this is an old version of this entry, which may differ significantly from the current revision.

The pandemic period has made remote work a reality in many organizations. Despite the possible negative aspects of this form of work, many employers and employees appreciate its flexibility and effectiveness. Therefore, employers are looking for the most optimal tools to support this form of work. However, this may be difficult due to their complexity, different functionality, or different conditions of the company’s operations. Decisions on the choice of a given solution are usually made in a group of decision makers. Often their subjective assessments differ from each other, making it even more difficult to make a decision. 

  • remote work
  • digital transformation
  • multi-criteria
  • tools

1. Introduction

Lockdown has changed the perception of working methods in many jobs that do not require the physical presence of an employee. If given such an opportunity, a significant number of companies took advantage of the opportunity to work remotely. For example, in the United States, the share of work from home increased from 14.4% to 39.6% of total employment between February and May 2020 [1]. At that time, about 70% of employees who could work from home decided to do so [2]. A year later, in June 2021, this rate dropped to 28.5%, which is about twice as high as before the pandemic [1]. The importance of remote work has also definitely increased in other countries worldwide, although this increase has not been uniform. In Germany, in April 2020, approx. 26% of employees worked entirely from home, and approx. 35% combined remote work with their presence in the workplace [3]. In France, this rate was around 25% in the same period, slightly higher in Italy [4], and, for example, in Japan, this rate was lower, reaching around 17% in June 2020 [5].
The pandemic and the lockdown have forced the acceleration to transform the employment models so that they comply with the rules of remote work. Moreover, in most countries worldwide, there was a lack of legal regulations adapted to the new situation, such as those concerning the organization and control of work (providing a workplace by the employer, occupational health and safety) or employment conditions [6,7,8]. The massive transition to remote work caused by the pandemic revealed both the advantages and limitations of such functioning [9]. A broad discussion began, covering many aspects that previously were not so important for both employees and employers. The new conditions turned out to be beneficial for a part of the society, they were positively received, but for another part they were not necessarily so. Difficulties in separating work from personal life can be a big problem. The lack of a clear separation of these two spheres of life may cause an imbalance between personal (family) life and work. An important, negative aspect affecting this balance may be (sometimes enforced by employers) the need for continuous availability via ICT (Information and Communications Technology). Work responsibilities can overshadow other activities of daily life, which in turn can negatively affect family relationships. Remote work therefore requires strong self-discipline, effective time management and the ability to focus on professional tasks at set times. Working from home can also lead to reduced interaction with other people and social isolation. The lack of direct meetings with co-workers may have a negative impact on developing mutual relations and solving professional problems. Remote communication tools (e-mails, messengers, videoconferencing) are not always as effective in conveying information as direct verbal and non-verbal communication. Another problem reported by employers is the difficulty in monitoring the performance of remote employees. Without direct supervision, in many cases it is difficult to assess the effectiveness of such work [10,11,12]. Due to many factors that may have a negative impact on remote work, this form of work may be difficult to accept for a part of society.
Other people, on the other hand, like this form of professional activity. They praise the convenience, flexibility of working hours, less stress [13,14], and in many cases they notice greater productivity [15,16,17]. Another frequently cited benefit is saving time and money for commuting to work or school [18,19] and less congestion on the roads [1,20]. In addition, as research [21,22,23,24] shows, a significant proportion of employees and employers were interested in continuing to work from home after the end of the pandemic.
An important benefit of the massive introduction of remote work during the pandemic is the reduction of greenhouse gas emissions, directly related to the so-called digital sustainability. With fewer commuters, car use and demand for public transport have decreased. This means less use of natural resources such as oil and coal, which contribute greatly to the greenhouse effect [25]. It is estimated that the mere reduction of commuting resulting from remote work could reduce CO2 emissions from land transport by approximately 8.5% per year [24]. In addition, due to remote work, offices and other facilities were able to operate at reduced power consumption, instead of fully functioning during working hours [25]. This is due to the fact that a more extensive and efficient infrastructure is used in corporate premises (e.g., lighting systems, professional printers, photocopiers, computers with large monitors, power failure protection systems, etc.). In addition, workstations are reorganized, for example by moving employees to vacant rooms. This allows for more optimal management of housing resources, and thus reduces the energy used for lighting, ventilation, air conditioning, heating systems, etc. These and other activities related to the pandemic and the introduction of remote work resulted in a further reduction of CO2, NO2 and other gas emissions, and thus improved air quality [26,27]. Therefore, it can be said that remote work affects digital sustainability. Although in the case of digitization of work, environmental issues may seem negligible, the introduction of remote work is associated with reducing pollutant emissions from transport and reducing energy consumption, which is environmentally friendly and has a positive impact on sustainable development.
In the context of introducing and continuing the digital “work from home” model, the availability of software that ensures efficient communication and performance of professional responsibilities via the ICT systems is very important. Among this type of software, teleconferencing systems such as Microsoft Teams, Zoom, Google Meet (formerly Hangout) and Skype [28,29] play the most significant role. These systems differ both in the range of functions offered, as well as in the costs of use. In addition, many users may have different feelings about using these systems.
Solving the problem of selecting an ICT system for the needs and conditions of a given organization has become the main motivator of the conducted research. Proposing an appropriate methodological solution that would facilitate the assessment of existing systems supporting remote work could be a valuable contribution to improving the process of selecting these systems. The methodological approach proposed by the authors fills a research gap. It is related to the integration of several important factors regarding the procedure for making decisions about the implementation of the system. One of them is support for group decision-making and reaching consensus among decision-makers. Another concerns the possibility of expressing the assessment not only numerically, but also using uncertain qualitative data. This type of data, presented using a linguistic scale, may turn out to be more effective than quantitative data with many evaluation criteria.
Multi-Criteria Decision Analysis (MCDA) and related methods deal with the issues of evaluation and support for the selection of considered variants. There are many of these methods and they can be grouped according to various criteria. One of them divides methods into those based on the outranking relation and on the utility function. The most popular methods from the first group are: ELECTRE methods [30], PROMETHEE methods [31], NAIADE [32], MELCHIOR [33], ORESTE [34], REGIME [35] and TACTIC [36]. The second group includes such methods as: AHP [37,38,39], ANP [40,41], DEMATEL [42], MAUT, REMBRANDT, MACBETH [43], SMART [44]. There are also many variants of multi-criteria methods (e.g., fuzzy), existing methods are integrated with each other and new methodological solutions appear, e.g., PVM [45,46], D-FTOPSIS [47], TROOIL [48], q-ROFDOSM–q-ROFWZIC [49], FAHP–TOPSIS, FAHP–VIKOR, FAHP–ELECTRE, FAHP–PROMTHEE [50].
PROSA GDSS is also a method of this type [51]. It is an MCDA method designed to solve decision problems by a group of experts. PROSA GDSS at the stage of individual assessment allows to apply, e.g., the NEAT F-PROMETHEE fuzzy method [52,53], thanks to which the uncertainty of assessments can be taken into account. In addition, the GAIA procedure was developed for the PROSA GDSS method, allowing for a graphical presentation and analysis of the preferences of individual decision-makers and the relationships between them.

2. An Overview of Teleconferencing Systems Supporting Remote Work and Criteria for Evaluating Systems

The COVID-19 pandemic has revolutionized the way we work. Thanks to the development and adaptation to teleconferencing platforms, it has become possible to perform many daily professional duties without leaving home [54,55,56]. Teleconferencing systems adapted to remote work should be characterized by appropriate functionalities, making these systems useful for the exchange of information and communication, both synchronous and asynchronous.
The most important functionality of teleconferencing systems used in remote work is to ensure synchronous communication with other users. Such systems allow for conducting employee videoconferences and business meetings. Teleconferencing platforms usually also provide functions for screen sharing, meeting recording, creating virtual rooms, file sharing, scheduling meetings in the calendar, exchanging text messages in the chat, etc. [57,58].
In situations where it is required to exchange files between users (especially large files), tools for storing and managing data in the cloud are used (e.g., Google Drive, OneDrive, Dropbox). Such solutions are safer than e-mail (multi-factor authentication), provide a large disk space and allow to improve teamwork [59]. Another aspect of remote work is team design and content creation, both textual and graphic, as well as various types of analytical and presentation studies. Popular tools include Google Docs, Microsoft 365, Visme [60,61].
Working in teams, especially dispersed ones, may also require tools to improve the management of groups working on joint projects. Dedicated tools for this purpose make it possible to increase productivity and reduce the inefficiency of remote teams [62]. Examples of software in this category include Slack, GanttPRO, Trello, ProProf Project.
Table 1 presents selected parameters of the most popular teleconferencing platforms supporting remote work.
Table 1. Selected parameters of teleconference platforms supporting remote work.
Taking into account the variety of teleconferencing systems and the functionality they offer, the selection of the appropriate platform supporting remote work should be carried out individually by each organization. However, based on the analysis of the literature, it is possible to determine a certain group of features and functionalities that should be met by the systems used. Thus, on the basis of expert knowledge, an initial selection of available tools can be carried out, thus supporting the choice of the system by organizations working remotely. The assessment of the considered platforms is made taking into account the criteria, the scope and number which may vary depending on the specific decision-making situation. They are usually adopted in the amount from a few to even a dozen, often grouped into thematic categories. Table 2 presents selected (based on the literature review) criteria for evaluating tools supporting remote work. These are the most frequently repeated criteria, but also those that have been recognized by the authors of the research as important due to the specificity of the analysed situation.
Based on the review of the research and the specified criteria for evaluating systems supporting remote work, it can be seen that their number is significant and they do not constitute a closed catalogue. Some of them are very detailed (e.g., face recognition attacks, calendaring), while others are formulated in a more general way (e.g., security, ease of use). However, they can be grouped into several thematic categories:
  • Functionality (number of participants allowed, file, content and screen sharing, application integrations, etc.);
  • Usability (ease of use, user interface, etc.);
  • Effective communication (chat support, video and audio feeds, etc.);
  • Multi-platform (multiplatform, mobile application, etc.);
  • Security and Privacy (malware attacks, face recognition attacks, confidentiality of personal data, security, etc.);
  • Technical performance (hardware requirements, network requirements, bandwidth consumption, communication delay, scalability, performance, quality of video and audio, etc.);
  • Support (support system, help desk, etc.);
  • Pricing (cost of software, cost of service, cost of equipment, etc.).
The presented criteria in many cases may be perceived by experts subjectively and immeasurable. For example, an expert familiar with a given teleconferencing platform may find it easier to use than the one he has not dealt with before. In addition, the ease of use cannot be measured, it can only be assessed qualitatively, of course, taking into account a certain margin of error and uncertainty. Therefore, in the assessment of teleconferencing systems supporting remote work, two aspects are important.
  • Capturing multi-expert ratings, aggregating those ratings, and exploring consensus among experts.
  • The ability to capture qualitative assessments, quantifiable and uncertain values.
The above requirements enforce the use of an appropriate multi-criteria assessment method. These requirements are met by combining the NEAT F-PROMETHEE and PROSA GDSS methods. The NEAT F-PROMETHEE method captures both quantitative and qualitative criteria expressed linguistically and of an uncertain nature. The PROSA GDSS method, on the other hand, makes it possible to take into account and aggregate the assessments of many experts, to study the preferences of decision-makers and the relationships between them, and to reward consensus and punish non-compliance of expert assessments and outliers. Therefore, a combination of the given MCDA methods was used in further studies.

This entry is adapted from the peer-reviewed paper 10.3390/su15139919

This entry is offline, you can click here to edit this entry!
Video Production Service