Multi-method Approach to Measuring Self-Regulated Learning: Comparison
View Latest Version
Please note this is a comparison between Version 2 by Lindsay Dong and Version 4 by Alenka Kavčič.

Self-regulated learning (SRL) is an active process in which individuals set goals, monitor their learning process, and regulate it according to goals and contextual demands. Most models of self-regulated learning assume that the purposeful use of specific processes, strategies, or responses is directed toward improving academic performance. They also assume that SRL involves cognitive, metacognitive, and motivational/affective processes, knowledge about these processes, and strategies for carrying them out. It is challenging to measure the (meta)cognitive and motivational/affective process and its components in SRL. No existing measure alone can capture the full complexity of this dynamic process and its contents. In general, SRL measures can be divided into offline and online measures. Offline measures, e.g., self-report questionnaires and interviews, attempt to capture self-regulation before or after the completion of the learning process, while online measures attempt to capture self-regulation in real time while learning is in progress.

  • self-regulated learning
  • learning strategies
  • science expository text
  • cognitive processes
  • metacognitive processes
  • motivational processes
  • offline measures
  • online measures
  • academic performance

1. Introduction

The COVID-19 pandemic has profoundly changed our lives. The use of ICT in education has increased [1]. Student individual learning and the use of electronic sources and expository texts in the e-environment has increased significantly [2]. Effective use of digital learning materials requires well-developed self-regulatory learning skills. Digital texts are usually presented in a non-linear format [3][4]; they contain images, animations, and various information search options. Therefore, compared to learning with printed materials, e-learning requires students to use more self-regulation during learning [5][6].
Although reading comprehension is higher when reading on paper than on screen [7], and student performance and reading times are higher when learning linear text than when learning hypertext [8], developing students’ self-regulated learning (SRL) improves learning efficiency in both traditional [9][10] and e-learning environments [11][12].

2. Self-Regulated Learning (SRL)

According to Zimmerman’s cyclical model [13], self-regulated learning (SRL) is a dynamic process that includes activities before, during, and after learning. The relationships between the processes in the phases of SRL are causal and cyclical. Before learning, motivational aspects (such as interest, task value, learner self-efficacy) influence learners’ decisions to set learning goals and plan learning activities. During learning, learners control themselves, the tasks, and the environment: they monitor their learning, selectively direct their attention, and use different strategies to remember and solve learning tasks. After learning, they undergo self-evaluation and make attributions for success or failure that trigger various positive or negative emotions (e.g., satisfaction, pride, shame, fear) that influence self-regulation in further learning. The processes in each phase affect processes in the next phase. Cognitive, metacognitive, and motivational/affective processes are constantly interwoven.

3. Measurement of SRL

It is challenging to measure the (meta)cognitive and motivational/affective process and its components in self-regulated learning (SRL). No existing measure alone can capture the full complexity of this dynamic process and its contents. In general, SRL measures can be divided into offline and online measures [14][15]. Offline measures, e.g., self-report questionnaires and interviews, attempt to capture self-regulation before or after the completion of the learning process. Such measures are subject to social desirability bias and students’ inability to access higher-order cognitive processes during learning due to memory failures, distortions, or interpretive reconstruction [15][16][17]. Online measures, on the other hand, attempt to capture self-regulation in real time while learning is in progress. Some of the online measures are unobtrusive and do not interfere with student learning [14], e.g., traces collected by computer software during learning (time spent on a particular page or content, clicks on hyperlinks, scrolling, and return to previous page). From these traces, inferences can be made about how students monitor and update their understanding or how much effort they put into learning [14]. On the other hand, the think aloud method can reveal students’ cognitive, metacognitive, and motivational processes, but it can be difficult for students to use and is not appropriate for studies that involve entire classes. Students can also take notes as they learn. These are a rather intrusive measure, but reveal a great deal about students’ self-regulation, particularly the learning strategies they use. Offline measures can be general or task-specific, whereas online measures are always task-specific [18].

4. Multi-method Approach

Studies of the convergent validity of different self-regulated learning (SRL) measures show inconsistent results. Correlations between offline measures vary from low to high [19][20][21], with most results in the low to moderate range. Results are also mixed for online measures. Some scholars report moderate to high correlations between different online measures, e.g., for strategies derived from writing journals [22] and from think aloud methods and systematic observations [23]. Research on correlations between off- and online measures also shows mixed results, ranging from low to moderate correlations between self-report questionnaires and notes, to no to moderate correlations between self-reports and the think aloud method [18], to high correlations between note-taking and think aloud protocols [24] and between self-reports and traces in the study material [25]. Low correlations between the off- and online measures of SRL suggest that they measure different aspects of SRL use in learning digital science texts. Several scholars [18][26][27] emphasized the importance of measuring self-regulation using multi-methods in order to avoid the drawbacks of using single measures and to gain valid insight into students’ actual approach to learning. Student achievement in science positively correlates with their use of SRL strategies. Studies show low to moderate correlations between offline measures of cognitive strategies and learning performance and moderate to large correlations for online measures [16][21]. Students with higher knowledge gains report the higher use of deep cognitive strategies, higher motivation for learning and use a higher number of strategies in note-taking while learning. Most off- and online SRL measures are positively related to student achievement, but correlations were low. In general, higher use of self-regulation in learning is associated with higher achievement [28][29][30][31][32][28,32,43,44,45]. Highly motivated students are more likely to use a variety of learning strategies in online courses [3328] and higher levels of strategy use positively correlates with higher learning achievement [3328]. Impact on student achievement was found for self-efficacy, effort regulation, increasing task value, improving attitudes toward the content, and time and study management [3429][3530]. All these rResults suggest the importance of cognitive, metacognitive, and motivational processes for better performance. Students with higher academic performance report greater use of deep cognitive strategies and metacognitive strategies, while their motivation is reflected in greater time spent learning, their higher initial motivation, and their engagement during learning. Deep cognitive strategies and motivation are more important than their initial knowledge of the topic. These findings have several practical implications for classroom practice. First, they suggest that teachers should help students learn metacognitive and deep cognitive strategies that promote higher levels of knowledge. An important part of promoting SRL is showing students how the effort they put into their individual learning, along with their engagement and time spent learning, translates into the acquisition of more organized knowledge and better grades.

References

  1. Pozo, J.-I.; Pérez Echeverría, M.-P.; Cabellos, B.; Sánchez, D.L. Teaching and Learning in Times of COVID-19: Uses of Digital Technologies during School Lockdowns. Front. Psychol. 2021, 12, 1511.
  2. Li, C.; Lalani, F. The COVID-19 Pandemic Has Changed Education Forever. This Is How. World Economic Forum. 2020. Available online: https://www.weforum.org/agenda/2020/04/coronavirus-education-global-covid19-online-digital-learning/ (accessed on 25 August 2021).
  3. Azevedo, R.; Cromley, J.G. Does Training on Self-Regulated Learning Facilitate Students’ Learning with Hypermedia? J. Educ. Psychol. 2004, 96, 523–535.
  4. Yen, M.H.; Chen, S.; Wang, C.Y.; Chen, H.L.; Hsu, Y.S.; Liu, T.C. A framework for self-regulated digital learning (SRDL). J. Comput. Assist. Learn. 2018, 34, 580–589.
  5. Azevedo, R. Computer Environments as Metacognitive Tools for Enhancing Learning. Educ. Psychol. 2005, 40, 193–197.
  6. Devolder, A.; van Braak, J.; Tondeur, J. Supporting self-regulated learning in computer-based learning environments: Systematic review of effects of scaffolding in the domain of science education. J. Comput. Assist. Learn. 2012, 28, 557–573.
  7. Kong, Y.; Seo, Y.S.; Zhai, L. Comparison of reading performance on screen and on paper: A meta-analysis. Comput. Educ. 2018, 123, 138–149.
  8. Blom, H.; Segers, E.; Knoors, H.; Hermans, D.; Verhoeven, L. Comprehension and navigation of networked hypertexts. J. Comput. Assist. Learn. 2018, 34, 306–314.
  9. Hattie, J.A.C. Visible Learning: A Synthesis of over 800 Meta-Analyses Relating to Achievement; Routledge: London, UK, 2009.
  10. Perry, V.; Albeg, L.; Tung, C. Meta-Analysis of Single-Case Design Research on Self-Regulatory Interventions for Academic Performance. J. Behav. Educ. 2012, 21, 217–229.
  11. Broadbent, J.; Poon, W.L. Self-regulated learning strategies & academic achievement in online higher education learning environments: A systematic review. Internet High. Educ. 2015, 27, 1–13.
  12. Kim, J.-K.; Kim, D.-J. Meta-Analysis on Relations between E-Learning Research Trends and Effectiveness of Learning. Int. J. Smart Home 2013, 7, 35–48.
  13. Zimmerman, B.J. Attaining Self-Regulation: A Social Cognitive Perspective. In Handbook of Self-Regulation; Boekaerts, M., Pintrich, P.R., Zeidner, M., Eds.; Academic Press: San Diego, CA, USA, 2000; pp. 13–39.
  14. Schraw, G. Measuring Self-Regulation in Computer-Based Learning Environments. Educ. Psychol. 2010, 45, 258–266.
  15. Veenman, M.V.J. Alternative assessment of strategy use with self-report instruments: A discussion. Metacogn. Learn. 2011, 6, 205–211.
  16. Dent, A.L.; Koenka, A.C. The Relation between Self-Regulated Learning and Academic Achievement Across Childhood and Adolescence: A Meta-Analysis. Educ. Psychol. Rev. 2016, 28, 425–474.
  17. Winne, P.H.; Jamieson-Noel, D. Exploring students’ calibration of self reports about study tactics and achievement. Contemp. Educ. Psychol. 2002, 27, 551–572.
  18. Rogiers, A.; Merchie, E.; Van Keer, H. What they say is what they do? Comparing task-specific self-reports, think-aloud protocols, and study traces for measuring secondary school students’ text-learning strategies. Eur. J. Psychol. Educ. 2020, 35, 315–332.
  19. Komarraju, M.; Nadler, D. Self-efficacy and academic achievement: Why do implicit beliefs, goals, and effort regulation matter? Learn. Individ. Differ. 2013, 25, 67–72.
  20. Mega, C.; Ronconi, L.; De Beni, R. What makes a good student? How emotions, self-regulated learning, and motivation contribute to academic achievement. J. Educ. Psychol. 2014, 106, 121–131.
  21. Veenman, M.V.J.; Bavelaar, L.; De Wolf, L.; Van Haaren, M.G.P. The on-line assessment of metacognitive skills in a computerized learning environment. Learn. Individ. Differ. 2014, 29, 123–130.
  22. Glogger, I.; Schwonke, R.; Holzäpfel, L.; Nückles, M.; Renkl, A. Learning strategies assessed by journal writing: Prediction of learning outcomes by quantity, quality, and combinations of learning strategies. J. Educ. Psychol. 2012, 104, 452–468.
  23. Veenman, M.; Cleef, D.V. Measuring metacognitive skills for mathematics: Students self-reports versus on-line assessment methods. ZDM 2019, 51, 691–701.
  24. Merchie, E.; Van Keer, H. Using on-line and off-line measures to explore fifth and sixth graders’ text-learning strategies and schematizing skills. Learn. Individ. Differ. 2014, 32, 193–203.
  25. Bråten, I.; Samuelstuen, M.S. Measuring strategic processing: Comparing task-specific self-reports to traces. Metacogn. Learn. 2007, 2, 1–20.
  26. Desoete, A. Multi-method assessment of metacognitive skills in elementary school children: How you test is what you get. Metacogn. Learn. 2008, 3, 189.
  27. Veenman, M.V.J. Learning to Self-Monitor and Self-Regulate. In Handbook of Research on Learning and Instruction; Mayer, R.E., Alexander, P.A., Eds.; Routledge: New York, NY, USA, 2011; p. 520.
  28. Dent, A.L.; Koenka, A.C. The Relation between Self-Regulated Learning and Academic Achievement Across Childhood and Adolescence: A Meta-Analysis. Educ. Psychol. Rev. 2016, 28, 425–474.Lin, C.-H.; Zhang, Y.; Zheng, B. The roles of learning strategies and motivation in online language learning: A structural equation modeling analysis. Comput. Educ. 2017, 113, 75–85.
  29. Veenman, M.V.J.; Bavelaar, L.; De Wolf, L.; Van Haaren, M.G.P. The on-line assessment of metacognitive skills in a computerized learning environment. Learn. Individ. Differ. 2014, 29, 123–130. Hattie, J.A.C.; Donoghue, G.M. Learning strategies: A synthesis and conceptual model. NPJ Sci. Learn. 2016, 1, 16013.
  30. Roebers, C.M.; Krebs, S.S.; Roderer, T. Metacognitive monitoring and control in elementary school children: Their interrelations and their role for test performance. Learn. Individ. Differ. 2014, 29, 141–149.Jackson, C.R. Validating and Adapting the Motivated Strategies for Learning Questionnaire (MSLQ) for STEM Courses at an HBCU. AERA Open 2018, 4, 2332858418809346.
  31. Lin, C.-H.; Zhang, Y.; Zheng, B. The roles of learning strategies and motivation in online language learning: A structural equation modeling analysis. Comput. Educ. 2017, 113, 75–85.
  32. Jackson, C.R. Validating and Adapting the Motivated Strategies for Learning Questionnaire (MSLQ) for STEM Courses at an HBCU. AERA Open 2018, 4, 2332858418809346.
  33. Lin, C.-H.; Zhang, Y.; Zheng, B. The roles of learning strategies and motivation in online language learning: A structural equation modeling analysis. Comput. Educ. 2017, 113, 75–85.
  34. Hattie, J.A.C.; Donoghue, G.M. Learning strategies: A synthesis and conceptual model. NPJ Sci. Learn. 2016, 1, 16013.
  35. Jackson, C.R. Validating and Adapting the Motivated Strategies for Learning Questionnaire (MSLQ) for STEM Courses at an HBCU. AERA Open 2018, 4, 2332858418809346.
More
© Text is available under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/)
Video Production Service
Feedback