Using Learning Gain to Evaluate Undergraduate Business Education: Comparison
Please note this is a comparison between Version 4 by Peter Tang and Version 3 by Martyn Polkinghorne.

During the COVID-19 pandemic, universities across the globe replaced in-person teaching, with online teaching, because of widespread campus shutdowns. Through the application of a pre-existing learning gain model, it was possible to review the learning of one cohort of students receiving in-person teaching, and then the next cohort of students on the same course receiving the online teaching. Overall, a drop in perceived learning was reported from the cohort of students receiving online teaching. For female students, the most significant drop in learning related to practical understanding, whereas for male students, the most significant drop in learning related to theoretical knowledge.

  • marketing education
  • student learning
  • learning gain
  • higher education
  • teaching quality
  • marketization
  • business education

1. Introduction

Students enrolling on undergraduate degrees now are unlike any generation that has preceded them. The population cohort commonly known as Generation Z (Gen Z) were born between 1995 and 2012 [1], and this group currently forms approximately 60% of the undergraduate university students in the UK [2]. They were born into a world where Google and Wikipedia already existed. They have never known a life without computers, or the internet, and do not see these technologies as tools, but as an integral part of their everyday existence [3]. They do not even view their mobile phones as being luxury items of technology, but instead consider them to be an essential part of everyday life [4]. These students are at home with the concept of social media, and use platforms such as Snapchat and Instagram for the majority of their communication. In fact, it is a generation that has garnered significant attention for its uniqueness in comparison with previous generations [3]. Together, these factors have a direct impact upon how we market the benefits and opportunities of higher education to them.
Simultaneously, university internationalization has become an important investment for education institutions in recent decades [5[5][6],6], and this has seen the higher education sector become an active player in the global marketplace [7]. Whilst this increased number of overseas students in the UK brings much added value to ongoing issues, such as classroom diversity and tuition fee income [7[7][8],8], it further challenges the operational ‘norms’ of pedagogy, with student demands for more technological interaction and experiential learning becoming more common.
Concerted effort to increase academic adoption and integration of digital, and social media, technologies to support contemporary pedagogic practice has been evident across the sector [3]. However, whilst the landscape of higher education has embraced advancements in technology-enhanced learning (TEL), to meet the needs of the 21st-century learner, students from this generation themselves were certainly not prepared to, or expecting to, experience their undergraduate degrees entirely through online learning, which has been the primary resulting impact for them of undertaking higher education during a pandemic which forced university campuses in the UK to close for approximately 18 months.
The shift from didactic face-to-face time with students to a complete program of online learning happened literally overnight for the higher education sector, as social distancing restrictions were put into place by the government, and new terminologies, such as ‘lockdown’ and ‘the tier system’, became the norm. Educators duly faced a number of social, pedagogical, and technological challenges in this new era of learning remotely through online teaching [9]. As a result, educational institutions continue to experience unprecedented times as they navigate students’ expectations with a range of TEL tools available, while balancing this with the reality of their own institutions’ ability to deliver a high-quality educational experience for students.

2. Online Teaching Pedagogies

Takavarasha [11][10] asserts that technology is now an integral part of everyone’s lives. The 21st-century consumer is a digitally ‘savvy’ choice maker [12][11] whose expectations, motives, and experiences are socially constructed [13][12]. This is evident in higher education, where the weresearchers witness innovative technology-enhanced teaching and learning models, often driven by student demands for bespoke learning [14][13]. Technology can positively impact student learning by offering a personalized and interactive experience, thereby enabling students to take a deeper responsibility for their educational journey [15][14]. Clearly, successful teaching strategies can minimize any resulting learning anxiety [16][15]. Furthermore, these creative approaches can be adopted by higher education institutions in an attempt to build synergy with the latest trends in educational technologies [17][16]. The authors acknowledge that shifts in education, and the introduction of technology-enhanced learning, were ongoing before the arrival of the COVID-19 pandemic. However, the campus closures due the pandemic acted as a catalyst for demanding rapid change which might otherwise have taken a decade or more to achieve. As a sector, we have therefore seen a shift from didactic teaching approaches in higher education to embracing creative pedagogies centred upon enhancing the student learning experience [18][17], with tactics such as flipped classroom (students receive teaching materials beforehand so that subsequent lecture time can be used to reinforce learning) and gamification (the use of game mechanics, such as points scoring and league tables within teaching to increase student engagement) now more commonly applied. This approach complements the characteristics of the modern-day student, who demands speed, nonlinear processing and social learning [19][18], collaboration [20][19], and multitasking [21][20].

3. Learning Gain

With the ever-increasing costs of higher education tuition fees [22,23,24][21][22][23], and an opposition to student loan funding support [25][24], the value of higher education is often questioned by students [26,27][25][26] and there is growing pressure to ensure the availability of relevant metrics to support the ongoing marketization agenda [6]. As a result, educators must consider the value and impact of their own teaching [28,29[27][28][29][30],30,31], whether it is delivered face-to-face, online, or as a hybrid of both. The landscape of higher education has therefore experienced a shift which has seen more measurement and assessment of student learning outcomes. This reliance upon metrics forms part of a performativity agenda [32][31]. Within such a metric-driven environment, the learning gains of students are now considered to be key indicators of teaching and learning excellence [33][32], and whilst other metrics may only feed into the cynicism discourse (moving to a consumerist model), and the pragmatism discourse (demonstrating value to stakeholders) of teaching excellence, evaluating student learning gain both supports these, and also supports the aspirationalism discourse relating to the integrity of teaching enhancement undertaken [34][33]. A recent office for students study concluded that existing methods for determining student learning gain need more development to enable them to account for contextual and subject factors [35,36][34][35]. Andrade [37][36] proposes that the actual definition of learning gain should be for individual higher education institutions to determine, but Arico et al. assert that ‘learning gain is now prominent when considering the effectiveness of higher education’ [38][37] (p. 249). As such, our understanding of student learning gain should help us to understand how we can maximize the effectiveness of our own teaching [29][28], and how the value of our teaching may have changed with the transition from face-to-face to online teaching. Taking these considerations into account, research has been undertaken which used an alternative learning gain model first proposed by Polkinghorne et al. [39][38], and subsequently detailed by Polkinghorne et al. [40][39], which integrates both student explicit knowledge (subject learning that can be codified and verbalized as theory) and tacit understanding (experience and practical application of knowledge) to create a unique two-dimensional evaluation of student learning. Therefore, the model is suited perfectly for holistically assessing learning outcomes, and for informing the continuous improvement process [41][40] to ensure that the quality of teaching being delivered, and the resulting student learning experience, can be understood, and then improved upon year on year.

4. Using Learning Gain to ResultsEvaluate Undergraduate Business Education

The learning gain model was applied to cohort of business studies students at a UK university who received in-person teaching, and the following year (during the pandemic) the same learning gain model was applied to the next cohort of business studies students on the identical course, but this time the students were receiving online teaching. Overall, the perceived learning reported by the students who received online teaching was lower. From the results of this restudyearch, it was of particular interest that the learning gain model used enabled the academic team to appreciate, for the first time, that in the case of the female students, the practical side of their education was perceived to be more affected by the change to online teaching, whereas the male students reported that the more theoretical side of their learning was perceived to be more affected. Such gender variations need now to be confirmed by further studies, but if true, this may have significant implications for how we teach online to ensure equity and inclusion. Such a potential differentiation in learning based upon gender needs to be accounted for in the online and hybrid models of university teaching currently being used across the global higher education sector.

References

  1. Singh, A. Challenges and issues of generation Z. J. Bus. Manag. 2014, 16, 59–63.
  2. Higher Education Statistics Agency (HESA). Available online: https://www.hesa.ac.uk/news/25-01-2022/sb262-higher-education-student-statistics/numbers (accessed on 3 March 2022).
  3. Seemiller, C.; Grace, M. Generation Z: A Century in the Making; Routledge: Abingdon, UK, 2018.
  4. Dhinakaran, V.; Partheeban, P.; Ramesh, R.; Balamurali, R.; Dhanagopal, R. Behavior and characteristic changes of generation z engineering students. In Proceedings of the 6th International Conference on Advanced Computing and Communication Systems (ICACCS), Coimbatore, India, 6–7 March 2020; pp. 1434–1437.
  5. Zhu, H.; O’Sullivan, H. Shhhh! Chinese students are studying quietly in the UK. Innov. Educ. Teach. Int. 2020, 1–10.
  6. Polkinghorne, M.; Roushan, G.; Taylor, J. Considering the marketing of higher education: The role of student learning gain as a potential indicator of teaching quality. J. Mark. High. Educ. 2017, 27, 213–232.
  7. Altbach, P.; Reisberg, L.; Rumbley, L. Tracking a global academic revolution. Change Mag. High. Learn. 2010, 42, 30–39.
  8. Gill, S. Overseas students’ intercultural adaptation as intercultural learning: A transformative framework. Compare 2007, 37, 167–183.
  9. Chunta, K.; Shellenbarger, T.; Chicca, J. Generation z students in the online environment: Strategies for nurse educators. Nurse Educ. 2021, 46, 87–91.
  10. Takavarasha, S. Digital development for a progressive networked society. Electron. J. Inf. Syst. Dev. Ctries. 2020, 86, 1–6.
  11. Bolat, E.; O’Sullivan, H. Radicalising the marketing of higher education: Learning from student-generated social media data. J. Mark. Manag. 2017, 33, 742–763.
  12. Kandiko, C.; Mawer, M. Student Expectations and Perceptions of Higher Education; King’s College London: London, UK, 2013.
  13. Johnson, L.; Adams Becker, S.; Estrada, V.; Freeman, A. NMC Horizon Report: Higher Education Edition; The New Media Consortium: Austin, TX, USA, 2015.
  14. Clark, D. Artificial Intelligence for Learning; Kongan Page: London, UK, 2020.
  15. Cheng, X. Asian students’ reticence revisited. System 2020, 28, 435–446.
  16. Kukulska-Hulme, A. How should the higher education workforce adapt to advancements in technology for teaching and learning? Internet High. Educ. 2012, 15, 247–254.
  17. Garrad-Cole, F.; Robinson, R.; Roberts, H.; Saher, M.; Ervine, J.; Donaldson-Hughes, C. Building Approaches to Learning in Online and Blended-Learning Environments: Challenges and Opportunities; Advance HE: York, UK, 2020.
  18. Rideout, V.; Foehr, U.; Roberts, D.; Generation, M. Media in the Lives of 8–18 Year-Olds; Henry, J., Ed.; Kaiser Family Foundation: San Francisco, CA, USA, 2021.
  19. Rosen, Y.; Beck-Hill, D. Intertwining digital content and a one-to-one laptop environment in teaching and learning: Lessons from the time to know program. Res. Technol. Educ. 2012, 44, 225–241.
  20. Teo, T. An initial development and validation of a digital natives assessment scale (DNAS). Comput. Educ. 2013, 67, 51–52.
  21. Callender, C.; Jackson, J. Does the fear of debt constrain choice of university and subject of study? Stud. High. Educ. 2008, 33, 405–429.
  22. Temple, P.; Callender, C.; Grove, L.; Kersh, N. Managing the student experience in English higher education: Differing responses to market pressures. Lond. Rev. Educ. 2016, 14, 33–46.
  23. Tomlinson, M.; Kelly, P. A prize for a price? HE marketization and the question of value. Theory Res. Educ. 2018, 16, 351–367.
  24. De Gayardon, A.; Callender, C.; Green, F. The determinants of student loan take-up in England. High. Educ. 2019, 78, 965–983.
  25. Chapleo, C.; O’Sullivan, H. Contemporary thought in higher education marketing. Mark. High. Educ. 2017, 27, 159–161.
  26. Roohr, K.; Liu, H.; Liu, O. Investigating student learning gains in college: A longitudinal study. Stud. High. Educ. 2017, 42, 2284–2300.
  27. Cameron, A.; Wharton, Y.; Scally, J. An investigation into the comparative learning gain and ‘value added’ for students from widening participation and non-widening participation groups: A case study from sports degrees. High. Educ. Pedagog. 2018, 3, 83–102.
  28. Evans, C.; Kandiko Howson, C.; Forsythe, A. Making sense of learning gain in higher education. High. Educ. Pedagog. 2018, 3, 1–45.
  29. Liu, O.; Liu, H.; Roohr, K.; McCaffrey, D. Investigating college learning gain: Exploring a propensity score weighting approach. J. Educ. Meas. 2016, 53, 352–367.
  30. Wood, M.; Su, F. What makes an excellent lecturer? Academics’ perspectives on the discourse of ‘teaching excellence’ in higher education. Teach. High. Educ. 2017, 22, 451–466.
  31. Skelton, A. Understanding Teaching Excellence in Higher Education: Towards a Critical Approach; Routledge: Abingdon, UK, 2005.
  32. Gunn, A. Metrics and methodologies for measuring teaching quality in higher education: Developing the teaching excellence framework (TEF). Educ. Rev. 2018, 70, 129–148.
  33. Gunn, V.; Fisk, A. Considering Teaching Excellence in Higher Education: 2007–2013; A Literature Review since the CHERI Report 2007; Higher Education Academy: York, UK, 2013.
  34. Jones-Devitt, S.; Pickering, N.; Austen, L.; Donnelly, A.; Adesola, J.; Weston, A. Evaluation of the National Mixed Methods Learning Gain Project (NMMLGP) and Student Perceptions of Learning Gain; Report to the Office for Students; Sheffield Hallam University: Sheffield, UK, 2019.
  35. Kandiko Howson, C. Final Evaluation of the Office for Students Learning Gain Pilot Projects; Report to the Office for Students; King’s College: London, UK, 2019.
  36. Andrade, M. Learning gain—A U.S. perspective. High. Educ. Pedagog. 2018, 3, 46–48.
  37. Arico, F.; Gillespie, H.; Lancaster, S.; Ward, N.; Ylonen, A. Lessons in learning gain: Insights from a pilot project. High. Educ. Pedagog. 2018, 3, 249–265.
  38. Polkinghorne, M.; Roushan, G.; Taylor, J. Evaluating student learning gain: An alternative perspective. In Proceedings of the Higher Education Academy (HEA) Surveys Conference—Understanding and Enhancing the Student Experience, Manchester, UK, 11 May 2017.
  39. Polkinghorne, M.; O’Sullivan, H.; Roushan, G.; Taylor, J. An innovative framework for higher education to evaluate learning gain: A case study based upon the discipline of marketing’. Stud. High. Educ. 2021, 46, 1740–1755.
  40. Polkinghorne, M.; Taylor, J.; Roushan, G. Continuous improvement in education: Understanding the effectiveness of our business and management. Bus. Manag. Chang. 2021, 19, 4–19.
More