- Please check and comment entries here.
Application of Maturity-Models in Universities
A maturity model is a widely used tool in software engineering and has mostly been extended to domains such as education, health, energy, finance, government, and general use. It is valuable for evaluations and continuous improvement of business processes or certain aspects of organizations, as it represents a more organized and systematic way of doing business.
We present a Systematic Literature Review study of the elements and purposes of the maturity models applied to universities. Obtaining results that show the intention to evaluating the maturity of only some areas of the universities and not the whole, creating some gaps that do not allow to measure the desired level. It also shows how information technologies help its implementation in the field.
According to , based on the interactions of people and the differentiation of their roles, defined the university as an organization of the type "professional bureaucracies." A bureaucracy for Max Weber is the efficient organization par excellence and that it defines even in the smallest details how things should be done . That is to say, the universities mostly regulate their work, as is the case for the selection of their teachers, who assume significant control over their own work, in the teaching-learning process. Since universities are organizations, maturity models have proven to be valuable in evaluating their processes, and determine by levels the path for academic excellence. Maturity models had their origin in the field of quality proposed by Philip B. Crosby, and were consolidated in the field of software engineering . The model "Quality Management Maturity Grid" is divided into five (5) progressive stages (uncertainty, awakening, enlightenment, wisdom, and certainty) and six (6) measurement categories (management’s attitude and understanding, organizational quality situation, problem management, cost of quality as a percentage of sales, actions for the improvement of quality and summary of the position of the company with respect to quality) that show the experience relationships that an organization has to go through. The form of evaluation proposed by Crosby is at the organizational, division or area level, and must be verified by three individuals: the quality manager of the operation, the general manager of the operation and a staff member who is not assigned to evaluation area, in order to cross opinions and establish an organization at a maturity level. This model recognizes the importance of human factors, such as leadership, attitude, and collaborative work .
Currently, the maturity models have been extended to different domains such as education, health, energy, finance, industrial sector, government and general use . Moreover, a maturity model is a widely used tool in evaluating fundamental processes or certain aspects of organizations, since it represents an increasingly organized and systematic way to do business .
Our study generates new knowledge proposals in the field of university education maturity, considering studies and prospects from the software engineering domain.
Twenty three of the 27,289 relevant articles were selected, with titles oriented towards university education and maturity levels. The remaining 1648 articles incorporated concepts of maturity in their purposes, but were very generic and did not define a method for evaluating maturity between levels and are oriented towards software development. The challenge will be to establish a flexible maturity model that contemplates the particularities of purposes defined by Universities, in order to diminish the impact of isolated maturity initiatives.
The selected articles corresponded to nine categories: oriented towards teaching, ICT, student monitoring, intellectual capital, e-learning, university entrepreneurship, employability of graduates, strategic planning of universities and IT governance in university institutions because no criteria were found for grouping articles in the literature review. The final 23 articles were analyzed according to their purpose, results and conclusions. We established nine categories and provided an overview of the specific domains of universities.
Maturity models in university education have moved towards agility, automation through ontologies, and the semantic web. These characteristics are being incorporated in maturity models of the application scope of greater use, software engineering, with favorable results due to the use of ontologies, a term that explains the capture of knowledge regarding a specific domain of interest. Thus, in university education, maturity models must have the capacity to add value to society in the acquisition and transfer of knowledge, supported by technology, automation and the semantic web.
In the teaching-learning category, Reference  proposes a maturity model for curricular design of higher education institutions in Malaysia. Based on CMMI, it presents five levels of maturity with process areas for each level. Reference  presents a maturity model based on the capacity of academic processes, with a maturity scale divided into five levels (see Figure 1).
Figure 1. Maturity scale. Adapted from Silva and Cabral (2010).
Reference  developed a maturity model with the goal of an engineering program accreditation based on CMMI and knowledge management. The maturity model compares software engineering processes with accreditation processes. It also defines how to advance from one level to another. The proposal of the authors is based on maturity models from the software engineering field; they use CMMI processes to align them with training processes of study programs or universities. Maturity model designs must be viewed from the needs and perspectives of University interest groups, to define assessment mechanisms for teaching and learning with flexible components that respond to institutional goals.
In the category of maturity models oriented towards ICT use, Reference  proposes an eight level maturity model, focusing on the use of generic hardware and software as the lowest level necessary for the infrastructure of student formation. Likewise, Reference  proposes a maturity model for IT Outsourcing in universities based on ISO 20,000, ISO 38,500, ITIL v3 standards, and Control Objectives for Information and Related Technologies (COBIT) methodologies as best practices in IT governance, with five maturity levels. Levels 1, 2 and 4 are based on ISO 20,000 & ITIL, level 3 is based on ISO 20,000 & author contributions and level 5 is strictly the author’s contribution. Assessing the maturity in the use of ICT for training with an outsourcing environment means transferring management to third parties, allowing updated technologies and services, and avoiding complications with technological waste and inventories. For , institutions of higher education require that ICT be part of the daily tools in educational processes. On the one hand, they must address the needs of students who expect, require and essentially demand digital technologies in their learning process. These proposals ensure maturity in the use of hardware, software and connectivity resources to reduce operating costs and improve educational services.
In the category of maturity models oriented towards student monitoring, Reference  frames the generational model to a model of capacity for maturity. The generational model implies an evaluation of the institutional capacity to initiate and plan student participation practices in the university. The capacity for maturity model includes five levels, a specific practice relationship, processes and categories. In addition, the model proposes a greater understanding and contextualization of student commitment. Reference  presents the SESR-MM as a maturity model to strengthen commitment, success and student retention in universities. The model structure has five maturity dimensions in five categories: learning, support, sense of belonging, integration and resources, as well as 18 processes and practices, and focuses on the improvement of training processes. Maturity models have components for assessing student performance throughout their education, with curricular and extracurricular activities included, and must ensure the achievement of the competencies established in the graduation profile.
In the category of maturity models focused on Intellectual Capital (IC), Reference  proposes an Intellectual Capital Maturity Model (ICMM) for universities within a framework divided in phases of actions for change, based on current maturity levels of significant environments. The ICMM contemplates six maturity levels: data collection, IC measurement and management relates to unstructured knowledge management; it is originated by the action, experience and participation of the university community in a specific context. For , the teaching process in universities represents intellectual capital for a variety of activities characterized by a mental and intellectual nature as well as a series of other traditional activities. In other words, it defines the ability to achieve the competencies that are defined in the graduation profile.
In the category of maturity models for E-learning, Reference  proposes a framework for the continuous improvement of university level e-learning (OCQMM). The OCQMM is based on CMMI and describes the characteristics and requirements of five maturity levels. Its structure includes processes that contemplate practices evaluated by dimensions. Reference  presents a pilot study using a method to compare e-learning in university institutions. The pilot was designed to evaluate the operational feasibility of a method based on the e-Learning Maturity Model (eMM), which was developed at the University of Wellington, New Zealand, which in turn was derived from the widely accepted Carnegie Mellon capacity maturity model. Reference  also applied a maturity model for e-learning for universities based on CMMI. It includes four process categories: learning, development, coordination and evaluation. In addition, it includes five dimensions: delivery, planning, definition, management and optimization. For , most universities tend to increase their use of Web 2.0 technologies to offer online courses, which implies using videos. E-learning is a field that has taken force in university education due to its ease of use and access to ICT, in addition to multiple human activities that facilitate online training. Currently, it is necessary to define criteria for the assessment of capacity and orientation in student formation.
In the category entrepreneurship maturity model category, Reference  proposes a business maturity model. The model describes 13 factors and 45 sub factors grouped into a business management framework that includes processes, resources and contexts. It contemplates three maturity scales and describes evaluation mechanisms. According to their objectives, some universities need criteria in order to evaluate the degree of student and graduate entrepreneurship, contributing to society in terms of productivity, skills and innovation, as a point of evaluation.
In the category of a support maturity model for university graduates for students, Pažur and Divjak  present a model that assesses the maturity of practices that contribute to the employability of graduates of the technology career. The model was developed using a five-step approach, from identification of purpose, to validation, following the paradigm of design science and exploiting four case studies across Europe. The final maturity model contains 65 practices within four key process areas: 13 within strategic planning, 26 within curriculum design and delivery, 16 within student support, and 10 within provision of extracurricular activities. Additionally, a description of the capacity assessment criteria is provided at the five maturity levels for each practice.
The model presented in the category of maturity model oriented to strategic planning, there is the model presented by Kaufman Hall . This maturity model measures the performance of the strategy-oriented financial planning of universities. Likewise, it is based on four levels of maturity with its practices, which measure performance from the formulation of the annual budget to a budget with periodic reviews, which ensure establishing strategies that allow it to achieve its institutional purposes.
Finally, in the category of IT governance-oriented maturity models, in , they present a model that assesses maturity from the edge of service delivery processes, under the COBIT 4.1 framework. This has five maturity levels or indicators of process capacity: level 1-executed, level 2-managed, level 3-established, level 4-predictable and level 5-optimized of IT governance in university institutions under the model of COBIT 4.1 and contemplates the service delivery processes. Each level of maturity has continuous improvement practices in the context of the use of Information Technologies in universities. Additionally, Reference  proposes an integration of the BSC with COBIT 4.1 in universities for integration of IT with their organizational strategy. On the other hand, Reference  proposes a model that assesses digital maturity in Croatian universities, based on rubrics and qualifications. This model proposes five levels: basic, initial, e-Enabled, e-Confident and e-Mature, it also has an instrument that determines the level of maturity.
The models developed or adapted in the university environment are based on a very specific isolated domain, neglecting other key areas of university organizations.
Generally, university maturity models do not suggest areas of work and practices that allow evaluating the level achieved, in order to strengthen continuous improvement.
With this study, it contributes to the state of the art, providing a methodology for identifying maturity models for universities.
We presented the results of a systematic review of 23 academic articles found in digital libraries and indexed databases of great relevance in the scientific and academic field. Likewise, the classification of the studies by publication year is shown in the bibliometric analysis, where an increase in the number of relevant studies published between 2014 and 2020 can be noted.
The maturity models with greater applicability are those oriented towards university education, and those of less applicability towards university entrepreneurship, due to the diversity in the declaration of their objectives that direct their university work. For some universities, the declaration of their objectives is oriented towards teaching, while others focus on teaching with components of university entrepreneurship. These characteristics diverge in the graduation profile, curriculum and educational objectives of each study program and by university.
Universities must adopt descriptive and prescriptive or evaluative maturity models with agility and automation components in order to ensure quality and continuous improvement in education. These models will be developed with tools that support automation through ontologies and the semantic web in order to ensure knowledge management.
Maturity models in their distinct orientations (teaching, ICT and student monitoring) are focused on measuring the maturity of student training, face-to-face or in a virtual modality, with the establishment of criteria that measures the capacity of the formation processes.
The entry is from 10.3390/info11100466
- Mintzberg, H. The professional bureaucracy. Organ. Gov. High. Educ. 1979, 2, 50–70.
- Chiavenato, I. Introducción a la teoría general de la administración; McGraw-Hill Interamericana: Mexico City, Mexico, 2019.
- Tarhan, A.; Turetken, O.; Ilisulu, F. Business process maturity assessment: State of the art and key characteristics. In Proceedings of the 2015 41st Euromicro Conference on Software Engineering and Advanced Applications, Funchal, Portugal, 26–28 August 2015; pp. 430–437.
- Crosby, P.B. Quality Is Free: The art of Making Quality Certain; McGraw-Hill: New York, NY, USA, 1979.
- Storbjerg, S.H.; Brunoe, T.D.; Nielsen, K. Towards an engineering change management maturity grid. J. Eng. Des. 2016, 27, 361–389.
- Lee, D.; Gu, J.W.; Jung, H.W. Process maturity models: Classification by application sectors and validities studies. J. Softw. Evol. Process. 2019, 31, e2161.
- Proença, D. Methods and techniques for maturity assessment. In Proceedings of the 2016 11th Iberian Conference on Information Systems and Technologies (CISTI), Las Palmas, Spain, 15–18 June 2016; pp. 1–4.
- Thong, C.L.; Jusoh, Y.Y.; Abdullah, R.; Alwi, N.H. Application of Curriculum Design Maturity Model at Private Institution of Higher Learning in Malaysia: A Case Study. In IAENG Transactions on Engineering Technologies; Springer: Dordrecht, The Netherlands, 2013; pp. 579–590.
- Silva, D.F.A.; Cabral, R.B. Maturity model for process of academic management. In Proceedings of the 2010 International Conference on Information Society, IEEE, London, UK, 28–30 June 2010; pp. 444–448.
- García, V.H.M.; Giraldo, G.A.M.; Quintero, J.F.L. Model accreditation for learning in engineering based on knowledge management and software engineering. In New Advances in Information Systems and Technologies; Springer: Cham, Switzerland, 2016; pp. 79–88.
- Bass, J. An early-stage ICT maturity model derived from Ethiopian education institutions. Int. J. Educ. Dev. Using Ict 2011, 7, 5–25.
- García, V.V.; Vicente, E.J.F.; Aragonés, L.U. Applicability of the Maturity Model for IT Service Outsourcing in Higher Education Institutions. Governance 2007, 5, 41–50.
- Espinoza-Guzmán, J.; Zermeño, M.G.G. Maturity model for e-learning classroom, bimodal and virtual courses in higher education: A preliminary study. Int. J. Web Based Learn. Teach. Technol. (IJWLTT) 2017, 12, 19–31.
- Nelson, K.; Clarke, J.; Stoodley, I.; Creagh, T. Using a Capability Maturity Model to build on the generational approach to student engagement practices. High. Educ. Res. Dev. 2015, 34, 351–367.
- Clarke, J.; Nelson, K.; Stoodley, I. The place of higher education institutions in assessing student engagement, success and retention: A maturity model to guide practice. In Research and Development in Higher Education: The Place of Learning and Teaching (Volume 36)-Refereed Papers from the 36th HERDSA Annual International Conference; Higher Education Research and Development Society of Australasia, Inc.: Auckland, New Zealand, 2013; pp. 91–101.
- Žilvinas, G.S.S.E.P.; Leitner, M.K.H. An intellectual capital maturity model (ICMM) to improve strategic management in European universities: A dynamic approach. J. Intellect. Cap. 2015, 16, 419–442.
- Naser, S.S.A.; Al Shobaki, M.J.; Amuna, Y.M.A. Knowledge Management Maturity in Universities and its Impact on Performance Excellence Comparative study. J. Sci. Eng. Res. 2016, 3, 4–14.
- Gu, D.; Chen, J.; Pu, W. Online course quality maturity model based on evening university and correspondence education (OCQMM). In Proceedings of the 2011 IEEE 3rd International Conference on Communication Software and Networks, Xi’an, China, 27–29 May 2011; pp. 5–9.
- Petch, J.; Calverley, G.; Dexter, H.; Cappelli, T. Piloting a process maturity model as an e-learning benchmarking method. Electronic Journal of e-Learning 2007, 5, 49–58.
- Al-Ammary, J.; Mohammed, Z.; Omran, F. E-Learning Capability Maturity Level in Kingdom of Bahrain. Turk. Online J. Educ. Technol. Tojet 2016, 15, 47–60.
- Wiele, P.V.; Ribiere, V.; Ermine, J.L. Diagnosing Higher Education on Purposefulness: Introducing the Employability Development and Assessment Maturity Model (EDAMM). In Empowering 21st Century Learners through Holistic and Enterprising Learning; Springer: Singapore, 2017; pp. 177–188.
- Markuerkiaga, L.; Errasti, N.; Ochoa, C.; Arcelus, M. UNEK, an Academic Entrepreneurship Maturity Model for Technological Faculties. Dir. Organ. 2017, 61, 12–18.
- Pažur Aničić, K.; Divjak, B. Maturity Model for Supporting Graduates’ Early Careers Within Higher Education Institutions. SAGE Open 2020, 10, 2158244019898733.
- Unveils Strategic Financial Planning Maturity Model for Higher Education at NACUBO Annual Meeting. 2019. Available online: https://n9.cl/hulwc (accessed on 10 September 2020).
- Kosasi, S.; Yuliani, I.D.A.E. Maturity levels of academic information services of higher education using IT governance. In Proceedings of the 2017 4th International Conference on New Media Studies (CONMEDIA), Yogyakarta, Indonesia, 8–10 November 2017; pp. 56–61.
- Pawan, E.; Utami, E.; Yunita, S.; Hasan, P. Measurement of Maturity Level Higher Education Governance Using Balanced Scorecard (BSC) and COBIT 4.1. In Proceedings of the 2019 International Conference on Information and Communications Technology (ICOIACT), Yogyakarta, Indonesia, 24–25 July 2019; pp. 948–953.
- Đurek, V.; Kadoic, N.; Ređep, N.B. Assessing the digital maturity level of higher education institutions. In Proceedings of the 2018 41st International Convention on Information and Communication Technology, Electronics and Microelectronics (MIPRO), Opatija, Croatia, 21–25 May 2018; pp. 671–676.