The category citizen participation in the identification of urban problems was further divided into the following subcategories (Figure 5): (1) participatory reporting (i.e., use of technologies such as crowdsourcing to allow citizens to report on urban problems), 25 studies; (2) participatory sensing (i.e., crowdsensing applications supported on the use of sensors from citizens’ personal devices to acquire diverse parameters), seven studies; (3) citizen-centered data analysis (e.g., social media data mining aiming to identify urban problems), 14 studies; and (4) multiple approaches (i.e., integration of various approaches such as participatory reporting in conjunction with citizen-centered data analysis), 13 studies.

Participatory reporting aims to provide city authorities with a better understanding of problems faced by the citizens. Ten studies [96,97,99,101,107,132,144,145,149,153] related to participatory reporting did not focus on specific issues, but instead described general purpose participatory reporting applications. These applications allow citizens to report on various types of issues (e.g., roads conditions, waste, traffic conditions, accidents, or crime, among others) and have various specific features: ref. [101] presented MinaQn, a web-based system that allows city officials to create questionnaires about different topics citizens can then answer; the application provided by [132] requires authentication so that the citizens can receive feedback about the status of the issues they reported; in the application described by [97], citizens must be authenticated so that their reports can be checked for quality and timeliness; the study reported by [144] was focused on the engagement of the citizens by using gamification concepts (e.g., user levels, avatars or leaderboards); ref. [99] presented an analysis engine to aggregate and consolidate the collected data; and the studies reported by [107,145,153] integrated comprehensive mechanisms to allow citizens to access services provided by the authorities (e.g., administration, education, healthcare, paying taxes or filling applications) and receive updates about city issues and the status of the reports they made.

Fifteen studies ref. [93,94,100,104,109,110,112,117,129,133,137,138,148,150,162] developed participatory reporting applications focused on specific issues: maintenance and accessibility of public spaces, mobility in the urban space, illegal waste dumping, safety, problems faced by pilgrims, and social problems (Figure 5).

The maintenance of public spaces was addressed by seven studies [93,100,112,117,133,138,162]. The applications developed by these studies allow citizens to report problems they find for proactive management of public spaces [162]. Although the participatory reporting mechanisms are similar among these studies, it is possible to distinguish specific features: ref. [93] proposed an application to allow citizens to share photos of points of interest; ref. [138] presented an application developed for the city of Tangerang, Indonesia, to allow the citizens to make suggestions from ten different categories of city development; in addition to participatory reporting, the prototype described by [112] also serves as a digital library so that citizens can have better information about their surroundings; the application presented by [100] allows the citizen to check the status of any reported issue; the application described by [117] makes use of gamification to incentivize citizens to participate; and the application reported by [133], FixMyStreet, according to official data, has been accepted by 98% of the British Councils.

In terms of the accessibility of public spaces, refs. [109,137] presented applications that allow citizens to comment and review city locations according to their accessibility to people with mobility impairments.

The study reported by [110] was focused on mobility in the urban space and presented a serious game aiming to introduce people to the concept of electric mobility and convince them of its utility. The gaming approach was chosen in lieu of more traditional surveys to investigate how gamification can improve the receptiveness of the public.

Considering that illegal waste dumping can cause social and health problems, the application proposed by [148] aimed at the quick identification of illegal waste dumping based on citizen reports. The application uses the Ethereum blockchain to create a currency that can be gained by performing certain actions on the system (e.g., reporting or voting), which can then be exchanged for goods and services from sponsors.

Since quick and efficient reporting of incidents that threaten the safety of citizens is of paramount importance, ref. [94] presented a method to increase the quality of data collected from an interactive voice response system for the city of East London, South Africa, that allows citizens to call and report safety incidents. Moreover, ref. [150] presented an application that allows two types of reports, an emergency one, where citizens only need to press a button to send pre-defined messages with their locations, or if the citizens are not in danger, four categories’ reports (i.e., criminal activity, perceived danger, suspicious activities and other) can be written and sent.

Based on the various problems pilgrims can encounter during the Hajj (Muslim holy pilgrimage), ref. [104] presented an application to share data with the city’s government and services, both from sensors and from citizen inputs, which might be used in conjunction with existing city services to support the management of the pilgrims as well as city residents.

Finally, one study [129] aimed at identifying the problems that affect minorities. In this regard, a basic foundational and theoretical framework for crowd mapping was developed to be used to create a crowd-based smart map for disabled people.

Looking specifically at the second subcategory related to citizen participation in the identification of urban problems (i.e., participatory sensing [95,102,106,113,116,127,136]), a subset of studies [102,106,127,136] had specific aims, while another subset of studies [95,113,116] was focused on general-purpose applications to allow the creation of participatory sensing campaigns to obtain different types of data (Figure 5).

Three aims were identified in terms of special-purpose applications: acoustic pollution, city events, and urban mobility. Concerning acoustic pollution, ref. [136] presented a participatory sensing architecture that allows smart devices to become noise sensors for urban environments, while [102] described an application that might be used to provide several services to support city events (e.g., festivals or concerts) since it collects precise location information that can be used by city officials for better management during normal operations or during emergencies. Finally, two studies focused on understanding how citizens move around the cities for a better planning of the available transportation: ref. [106] presented a reference architecture for an application that collects data from mobile sources, analyzes the collected data and then gives feedback to the citizens, namely alerts of dangerous spots. The study reported by [127] used the ParticipAct platform (also used by the study reported by [109]) to store location data collected from the citizen mobile devices, which were then analyzed to infer the citizen mobility patterns (i.e., what paths they take from location to location) and identify points of interest in the cities.

Fourteen studies [108,118,125,135,141,142,143,146,147,152,155,156,157,158] proposed applications that analyze citizen-centered data from social media (e.g., Facebook or Twitter) to identify needs or problems faced by the citizens to support urban planning and to determine the citizens’ opinions about the performance of local authorities (Figure 5).

Seven studies [108,125,135,142,146,147,155] were related to the identification of needs and problems faced by the citizens. The study reported by [155] used NeedFull, a tweet analysis framework, to investigate the reactions of the people of New York State during the COVID-19 pandemic, while the remaining six studies [108,125,135,142,146,147] aimed to analyze large volumes of data to understand the sentiments of the citizens about certain topics [108,125,142,146,147] or to infer alerts, insights, or recommendations [108].

Concerning urban planning, one study [141] used freely available data on citizen activities to identify common points of interest and urban areas where sports are played. In contrast, another study [158] used a machine-learning algorithm to identify citizen trends in terms of urban planning by analyzing data from a civic participation application.

Moreover, five studies [118,143,152,156,157] aimed to identify the opinion of the citizens about the local authorities. Article [156] presented an analytical framework to retrieve citizen-centered data from an online comment board to be used by local governments to assess political reforms and implementations. In turn, refs. [152,157] presented how social media can be used to generate data that municipalities can use to implement smart cities. Furthermore, ref. [118] presented how governments can use social media to analyze specific services and better understand their citizenry’s opinion (i.e., positive, negative, or neutral) of those services. Additionally, ref. [143] presented a system that uses social media data to identify trending views and influential citizens.

Finally, looking specifically at the thirteen studies [98,103,115,123,126,128,134,139,151,161,164,166,168] that focused on the use of multiple approaches to identify urban problems (i.e., the fourth subcategory related to citizen participation in the identification of urban problems), two articles [103,161] presented generic tools (Figure 5): ref. [103] presented a middleware that allows the collection of data from multiple sources, be they static sensors, participatory sensing by citizens or data mined from social media, while [161] presented an application that allows citizens to report city problems and collect data through smartphone sensors. The remainder eleven studies [98,115,123,126,128,134,139,151,164,166,168] focused on specific issues: maintenance of the urban infrastructures [123,128,134,166], accessibility and mobility in the urban spaces [98,115,126,139], air and acoustic pollution [151], acoustic planning [168], and creation of a data repository [164] (Figure 5).

In what concerns the maintenance of urban infrastructures, namely the maintenance of roads potholes, ref. [166] presented an application that uses data mining of public data on Twitter together with data provided by a mobile application to allow citizens to report any potholes they find, including data about the hole (e.g., depth, diameter, or damage) along with its geolocation. In turn, ref. [123] presented an application that supported an urban data challenge to allow young citizens to document and reflect on their city problems through photos, videos, interviews, and posts from Facebook and Twitter. Moreover, the application presented by [128] aims to allow citizens to perform tasks to monitor urban services using participatory reporting and participatory sensing. Additionally, ref. [134] presented a unified framework to use different data sources, including data from social media and reports or measurements provided by the citizens.

Regarding accessibility and mobility in urban spaces, two studies [98,115] focused on the mobility of impaired people. Article [98] presented an application that uses participatory reporting, participatory sensing, and city open data to create tailored routes for citizens with mobility impairments, considering a routing algorithm that takes accessibility barriers as constraints. In turn, ref. [115] presented an application to promote urban accessibility by having citizens, both with and without disabilities, using wearable sensors to collect their movement patterns, which are processed to identify the routes that are not used by citizens with disabilities.

Still, in terms of mobility in the urban spaces, ref. [139] presented an application that aims to use both crowdsourced geodata from citizen reports and data gathered from Twitter to predict events that are likely to happen the next day in the same geographical area, while [126] presented a participatory sensing application to assess trip quality when riding in a vehicle. This application collects data from sensors and allows the citizens to report specific situations, and the aggregate data are analyzed to determine road or traffic quality [126].

Knowing that smart cities might reduce pollution levels if pollution sources are identified, ref. [151] presented an application to measure air pollution and noise levels in a city using multiple inputs: a network of high-precision static nodes, lower-precision mobile nodes, microphones of mobile devices to gather random noise samples, open access data sources, and citizen participation by answering questionnaires about air quality and noise levels. In turn, considering that the sounds of an area and how they impact people’s lives can be unintentionally neglected when designing urban environments, ref. [168] presented an application composed of multiple software tools to allow citizens to collect soundscapes and to provide reports to be used as part of the process of planning and designing urban environments, which is more commonly focused on the visual elements.

Concerning the creation of data repositories, ref. [164] proposed an application that allows various types of stakeholders (e.g., citizens, government, or companies) to collect data about a city from various sources (e.g., citizen reporting, dedicated sensors, or social networks), to be shared and visualized in several ways (e.g., 3D renders, heatmaps or lists).

When planning and designing urban environments, the citizens living and working in those environments are the most affected. Therefore, it is an objective for smart cities to have citizens involved in the governance processes. In this respect, different types of collaborative applications were identified (Figure 6) to (1) support citizen participation in urban planning and design; (2) provide visualization tools for urban planning; (3) allow participatory budgets; (4) allow the dissemination of the citizen ideas and prioritization of city projects considering citizen satisfaction; and (5) promote democratic values including the transparency of public administration.

Six studies [114,119,121,140,154,163] focused on collaborative applications for citizen participation in urban planning. Article [114] presented an application designed to give the common person the tools and knowledge necessary to create and orchestrate citizen participatory reporting and sensing campaigns based on a six phases process: identification of issues, framing those issues in the existing context, design, deployment, orchestration of the finished product, and outcome review. In turn, collaborative applications to support city decision-making were presented by [119,163]. Moreover, refs. [121,140] presented applications to allow citizens and government officials to share information between themselves, and [154] proposed an application for the co-creation of neighborhoods by allowing the collaboration between citizens, and between citizens and the municipalities for the design and approval of houses, public spaces, renting spaces or other situations where government officials would also need the collaboration of the citizens.

Since it can be hard to extract information from large volumes of data, four studies [120,122,131,167] focused on applications providing visualization tools to allow citizens, both experts and non-experts, to better understand the repositories containing data related to urban planning: ref. [122] presented a tool to facilitate the visualization of data collected by city governments on several topics (e.g., pollution, attractiveness of surroundings, or resource management), and to allow citizens to give feedback on the data presented and services available; ref. [131] presented the Quick Urban Analysis Kit, which aims at the design of urban spaces by the citizens167] presented a serious game to allow citizens to collaborate in constructing a smart city; and [120] presented a mixing panoramic imaging and architectural drawing tool for future urban plans.

Regarding participatory budgets, ref. [130] presented an application to allow citizens to propose projects to be funded and implemented. In turn, ref. [105] described an application to prioritize city projects considering citizen satisfaction according to various metrics. Finally, ref. [111] presented an application to allow citizens to present ideas for the benefit of the cities and comment and vote on ideas already posted. To incentivize citizen participation, competitions might be considered to distribute prizes such as gift certificates [111].

Four studies [124,159,160,165] focused on applications to promote democratic values and scrutinize public authorities’ decisions. Article [124] described the Visor Urbano application, which aims to lower city governments’ corruption by allowing citizens to request permits for building, opening businesses and other licenses. In turn, ref. [159] described an application to help identify misuse of resources and corruption in public works, allowing citizens to view the details of public works and report any inaccuracies or suspicious details they found to the government. Moreover, two articles [160,165] presented applications to provide the discussion and review of governmental policies: the applications for urban democracy that were implemented in Madrid and Barcelona were presented by [160]. In contrast, ref. [165] presented a government petition application of Taiwan that allows citizens to propose petitions if they reach the threshold of 5000 signatures in 60 days. Since the application has been in service since 2015, several proposed petitions have been implemented into policy.

In terms of data sources, 25 studies focused on participatory reporting [93,94,96,97,99,100,101,104,107,109,110,112,117,129,132,133,137,138,144,145,148,149,150,153,162] and in other 12 studies [98,115,123,126,128,134,139,152,161,164,166,168] participatory reporting was used together with other approaches, such as participatory sensing or citizen-centered data analysis.

In turn, 19 studies (Table 3) implemented participatory sensing applications that use the sensors from personal mobile devices to collect data related to location, activity, and environment.

Furthermore, fourteen studies [108,118,125,135,141,142,143,146,147,152,155,156,157,158] were exclusively focused on the analysis of social media data (e.g., Facebook or Twitter), while seven other studies [103,123,128,134,139,164,166] combined the analysis of social media data with participatory reporting and participatory sensing.

In terms of data sources, one article [98] referred to the use of open data about real-time public transportation means (i.e., available equipment and respective accessibility barriers and facilitators), while another article [151] referred the use of open access data sources and a network composed by high precision static nodes and lower precision mobile nodes in addition to the microphones of the citizen mobile devices and answers to questionnaires about air quality and noise levels.

Since different heterogeneous data sources were considered, there is the possibility of contradictory data or data with low quality (e.g., missing values or outliers). Therefore, analyzing data quality and minimizing the consequences of low-quality data are relevant processes to be considered. However, just four articles reported how low-quality data are managed: the low-quality or incomplete data are filtered out by a classifier [97]; the open data collected from various sources were cleaned to eliminate corrupt and duplicate data, and standardized into a homogenous format, with the sources themselves checked for consistency [122]; the most extreme five percent of data records (best and worst) were ignored during evaluations, and the same ride trips were compared to each other to identify and correct abnormal readings [126]; data collected from multiple sources were verified for validity, outlier detection and missing values [151].

The applications reported by the included studies can potentially put the participants’ privacy at risk, namely in terms of the communication of personal data. Therefore, secure data transmission and storage must be guaranteed. However, only 10 articles [94,102,107,115,116,126,128,139,150,151] referred data security and privacy mechanisms or the usage of security frameworks: in the study presented by [94] auditing mechanisms were used to reinforce access control mechanisms102] reported the implementation of a privacy module to allow the citizens to manage the data collection and to prevent unauthorized accesses; the study reported by [107] used OpenStack services and Spring Security framework115] presented a method to preserve anonymity with Radio Frequency Identification (RFID); in the study reported by [116] the data transmitted was encrypted using the Advanced Encryption Standard (AES)126] summarized several methods to handle privacy issues in participatory sensing applications; the study reported by [128] implemented a location obfuscation mechanism; in [139] is mentioned as a future work the mitigation of denial of service attacks; in the study reported by [150] the Secure Sockets Layer (SSL) protocol is used to encrypt data to be transmitted or stored; and [151] presented a method that allows recognizing environmental sounds while speech intelligibility is masked.

In addition to potential privacy risks, citizen participation requires consuming their resources, such as battery and computing power. Therefore, since the impact and relevance of participation campaigns depend on the engagement of the citizens, incentive mechanisms might be proposed, which can be either monetary or non-monetary (e.g., entertainment). To guarantee the engagement of the citizens, 19 of the included articles [95,97,101,103,110,111,112,113,116,117,126,128,132,144,145,148,151,160,167] reported the need to implement incentive mechanisms. However, only four of them [110,117,144,167] reported the use of gaming mechanisms and the other four [97,126,145,148] implemented incentive mechanisms considering different approaches: in [97], incentives were evaluated empirically to ascertain which one is better in which conditions to promote citizen participation; in [126], the various stakeholders gained benefits from the data collected; in [145], every performed action gained points for a ranking system, and in [148], the participants were awarded with a civic currency that can be traded by assets provided by sponsors.