Frameworks and Practices for Smart City Assessment: Comparison
View Latest Version
Please note this is a comparison between Version 2 by Lindsay Dong and Version 1 by Thajba Aljowder.

Frameworks and practices for smart city assessment belong to five main categories: best practices, ranking frameworks, index-based frameworks, initiative-based evaluation, and maturity models.

  • smart city
  • assessment framework
  • sustainability
  • maturity model

1. Introduction

Cities worldwide face population growth, rapid urbanization, improper resource management, and inadequate infrastructure. These challenges pose environmental threats, drain resources, weaken infrastructure and the economy, and trigger social problems such as high crime rates and inequality. To manage and mitigate these problems, cities are racing to digital transformations and to become smarter. According to the World Cities Report 2020 [1], the global demand for smart cities grew from USD 622 billion in 2017 to USD 1 trillion in 2019 and is forecasted to reach USD 3.48 trillion by 2026. The United Nations’ 2030 Agenda for Sustainable Development emphasizes the need for smart and sustainable cities globally, particularly the sustainable development goal (SDG) 11: “make cities and human settlements inclusive, safe, resilient and sustainable” [2] (p. 26). Successful smart city implementation will contribute to achieving the SDGs [3,4][3][4].
Smart city implementation not only helps governments provide more efficient services but also promotes innovation, encourages private–government partnerships, enhances decision-making processes, improves project financing, and promotes sustainability [5,6][5][6]. However, many technical, social, economic, and strategic challenges must be overcome to realize the benefits of the smart city concept. Contemporary cites are characterized by complexity, diversity, and intelligence [7], which can be barriers to smart transformation. Therefore, it is crucial to ensure the availability of reliable governance systems that can plan, manage, and measure smart transformation.
Academic studies and practitioners have provided a variety of conceptualizations of smart cities. The definition in [8] (p. 11) focuses on the main characteristics of a smart city: “a city well performing in a forward-looking way in the six characteristics (economy, people, governance, mobility, environment, living), built on the smart combination of endowments and activities of self-decisive, independent and aware citizens”. Other definitions emphasize the role of technology. For example, the International Data Corporation defines smart cities as cities that use ubiquitous networks, wireless sensors, and intelligent management systems to solve current and future challenges and create new services [9].
Standards bodies such as the British Standards Institution (BSI) adopt a holistic view of smart cities and the utilization of best practices when defining a smart city as “a city where there is effective integration of physical, digital and human systems in the built environment to deliver a sustainable, prosperous and inclusive future for its citizens” [10] (p. 18). The International Telecommunication Union Telecommunication Focus Group on Smart Sustainable Cities (ITU-T FGSSC) analyzed 120 definitions to develop a comprehensive definition of a smart city: “an innovative city that uses information and communication technologies (ICTs) and other means to improve quality of life, efficiency of urban operations and services, and competitiveness, while ensuring that it meets the needs of present and future generations with respect to economic, social, environmental as well as cultural aspects” [11] (p. 13).

2. Frameworks and Practices for Smart City Assessment

2.1. Best Practices

International, regional, and local standards bodies issue best practices to ensure the quality of services and enable cities to perform benchmarking. These bodies include the International Organization for Standardization (ISO), the International Electrotechnical Commission (IEC), the International Telecommunication Union (ITU), the European Committee for Standardization (CEN), the British Standards Institution (BSI), and the National Institute of Standards and Technology (NIST). These organizations have published standards that define a smart city and specify relevant indicators for assessment, such as ISO 37120:2018 (Sustainable cities and communities—Indicators for city services and quality of life), ISO 37153:2017 (Smart community infrastructures—Maturity model for assessment and improvement), ISO/IEC 30146:2019 (Information technology—Smart city ICT indicators), PAS 181:2014 (Smart city framework—Guide to establishing strategies for smart cities and communities), and ITU-T L.1603 (Key performance indicators for smart sustainable cities to assess the achievement of sustainable development goals). However, applying best practices in real-life projects can be challenging.

2.2. Ranking Frameworks

Ranking frameworks compare cities by using specific criteria, which allows cities to act based on their relative positions. The European smart cities ranking (ESCR) [8,20][8][12] is a very well known ranking framework. Giffinger and colleagues studied 58 medium-sized cities to define a ranking mechanism for European cities based on six main characteristics: society, environment, economy, governance, mobility, and living. They also identified 31 factors and 74 indicators that can be used to measure cities’ performance. A limitation of ranking systems is that they focus on the final results and do not capture enough detail about cities’ strengths and weaknesses. Moreover, these frameworks do not consider cities as complex and unique systems that require comprehensive yet specific measurements based on local context.

2.3. Index-Based Frameworks

Index-based frameworks use key performance indicators. The CITYKeys project, which is funded by the European Commission, has developed a indices-based performance measurement framework for monitoring smart city implementation [21][13]. A limitation of these frameworks is that they are subject to data availability.

2.4. Initiative-Based Evaluation

Initiative-based evaluation is similar to the framework developed by [22][14] to examine smart city initiatives based on a set of identified critical success factors. This framework includes management and organization, technology, governance, policy context, people and communities, economy, built infrastructure, and the natural environment and provides a comprehensive conceptualization of the smart city. However, initiative-based evaluation frameworks may encourage isolation and barriers between different sectors of smart city projects.

2.5. Maturity Models

A maturity model is an assessment tool for establishing the current situation and identifying necessary improvements to progress in maturity. Maturity models have been used as assessment tools in many sectors to ensure continuous improvement such as process management, project management, knowledge management, sustainability management, risk management, supply chains, education, government, construction, and healthcare [23][15]. Maturity models can be applied to measure the success of smart city adoption [24][16].
Maturity models comprise several components [25][17]:
  • The domain is the model’s first layer and provides a high-level view of the scope;
  • The domain components, sometimes referred to as focus areas or factors, are the significant aspects of the domain, such as critical success factors;
  • The domain subcomponents, also called capabilities or processes, provide further details. Achieving the capabilities will determine the level of maturity;
  • The levels can be present in any number, depending on the model scope and application. Maturity can range from the minimum value, i.e., the quality of the elements underlying the processes is in the lowest required state, to the maximum value, i.e., no further improvements are needed [26][18];
  • The assessment tool defines how the capabilities will be measured against the maturity scale using qualitative (descriptions) or quantitative (numerical scales) approaches. Assessment can either be self-assessment or performed by a third party.
Maturity models can be classified into two types based on stages, fixed levels, and focus areas. The fixed levels have a specific number of levels of maturity for all defined focus areas. A well-known fixed-level model is the CMM. It is popular and simple to implement but is not recommended for complex environments [23][15]. The focus area maturity model divides the functional domain into a number of focus areas that need to be developed to achieve maturity in the functional domain. Related focus areas can be grouped to facilitate the assessment. Each focus area includes (1) different capabilities that represent the steps of development of the focus area and (2) different numbers of maturity levels. The final maturity is a combination of the maturity levels of the focus areas.
The focus area maturity model has been used to develop models in disciplines such as enterprise architecture [27][19], information security [28][20], software governance [29][21], master data management, disaster risk management, and social media [30][22].

References

  1. United Nations Human Settlements Programme. World Cities Report 2020. The Value of Sustainable Urbanization. 2020. Available online: https://unhabitat.org/world-cities-report-2020-the-value-of-sustainable-urbanization (accessed on 15 November 2020).
  2. United Nations. Transforming Our World: The 2030 Agenda for Sustainable Development. Available online: https://sustainabledevelopment.un.org/content/documents/21252030%20Agenda%20for%20Sustainable%20Development%20web.pdf (accessed on 15 November 2020).
  3. Ismagilova, E.; Hughes, L.; Dwivedi, Y.K.; Raman, K.R. Smart cities: Advances in research—An information systems perspective. Int. J. Inf. Manag. 2019, 47, 88–100.
  4. Klopp, J.M.; Petretta, D.L. The urban sustainable development goal: Indicators, complexity and the politics of measuring cities. Cities 2017, 63, 92–97.
  5. Belanche, D.; Casaló, L.V.; Orús, C. City attachment and use of urban services: Benefits for smart cities. Cities 2016, 50, 75–81.
  6. Marchetti, D.; Oliveira, R.; Figueira, A.R. Are global north smart city models capable to assess Latin American cities? A model and indicators for a new context. Cities 2019, 92, 197–207.
  7. Fernández-Güell, J.-M.; Guzmán-Araña, S.; Collado-Lara, M.; Fernández-Añez, V. How to incorporate urban complexity, diversity and intelligence into smart cities initiatives. In Smart Cities. Smart-CT 2016; Alba, E., Chicano, F., Luque, G., Eds.; Lecture Notes in Computer Science; Springer: Berlin/Heidelberg, Germany, 2016; Volume 9704.
  8. Giffinger, R.; Fertner, C.; Kramer, H.; Kalasek, R.; Pichler-Milanović, N.; Meijers, E. Smart Cities-Ranking of European Medium-Sized Cities ; Centre of Regional Science, Vienna University of Technology: Vienna, Austria, 2007; Available online: https://www.smart-cities.eu/download/smart_cities_final_report.pdf (accessed on 1 December 2020).
  9. Clarke, R.Y. Smart Cities and the Internet of Everything: The Foundation for Delivering Next-Generation Citizen Services . IDC Government Insights. 2013. Available online: https://www.cisco.com/c/dam/en_us/solutions/industries/docs/scc/ioe_citizen_svcs_white_paper_idc_2013.pdf (accessed on 1 December 2020).
  10. British Standards Institution. PAS 180 Smart Cities. Vocabulary; BSI: London, UK, 2014.
  11. International Telecommunication Union. Smart Sustainable Cities: An Analysis of Definitions . 2014. Available online: https://www.itu.int/en/ITU-T/focusgroups/ssc/Documents/website/web-fg-ssc-0029-r14-overview_role_of_ICT.docx (accessed on 15 November 2020).
  12. Giffinger, R.; Haindlmaier, G. Smart cities ranking: An effective instrument for the positioning of the cities? ACE Archit. City Environ. 2010, 4, 7–26.
  13. Bosch, P.; Jongeneel, S.; Rovers, V.; Neumann, H.-M.; Airaksinen, M.; Huovila, A. CITYkeys Indicators for Smart City Projects and Smart Cities; European Commission and H2020 Programme; 2017.
  14. Hafedh, C.; Nam, T.; Walker, S. Understanding smart cities: An integrative framework. In Proceedings of the 2012 45th Hawaii International Conference on System Sciences, Maui, HI, USA, 4–7 January 2012.
  15. dos Santos-Neto, J.B.S.; Costa, A.P.C.S. Enterprise maturity models: A systematic literature review. Enterp. Inf. Syst. 2019, 13, 719–769.
  16. Liu, J.; Chen, N.; Chen, Z.; Xu, L.; Du, W.; Zhang, Y.; Wang, C. Towards sustainable smart cities: Maturity assessment and development pattern recognition in China. J. Clean. Prod. 2022, 370, 133248.
  17. de Bruin, T.; Freeze, R.; Kulkarni, U.; Rosemann, M. Understanding the main phases of developing a maturity assessment model. In Proceedings of the ACIS 2005 Proceedings, Jeju Island, Republic of Korea, 14–16 July 2005; Volume 109. Available online: https://aisel.aisnet.org/acis2005/109 (accessed on 14 February 2021).
  18. Gochermann, J.; Nee, I. The idea maturity model—A dynamic approach to evaluate idea maturity. Int. J. Innov. Technol. Manag. 2018, 16, 1950030.
  19. van Steenbergen, M.; van den Berg, M.; Brinkkemper, S. A balanced approach to developing the enterprise architecture practice. In Proceedings of the Enterprise Information Systems: 9th International Conference (ICEIS 2007), Funchal, Madeira, 12–16 June 2007; Revised Selected Papers 9. Springer: Berlin/Heidelberg, Germany, 2008; pp. 240–253.
  20. Spruit, M.; Roeling, M. ISFAM: The Information Security Focus Area Maturity Model. ECIS 2014 Proceedings. Available online: https://aisel.aisnet.org/ecis2014/proceedings/track14/6/ (accessed on 15 March 2021).
  21. Jansen, S. A focus area maturity model for software ecosystem governance. Inf. Softw. Technol. 2020, 118, 106219.
  22. Sanchez-Puchol, F.; Pastor-Collado, J.A. Focus area maturity models: A comparative review. In Proceedings of the Information Systems: 14th European, Mediterranean, and Middle Eastern Conference (EMCIS 2017), Coimbra, Portugal, 7–8 September 2017; Proceedings 14. Springer: Berlin/Heidelberg, Germany, 2017; pp. 531–544.
More
© Text is available under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/)
Video Production Service
Feedback