Deep Meaningful Learning: Comparison
Please note this is a comparison between Version 2 by Nora Tang and Version 1 by Stylianos Mystakidis.

Deep meaningful learning is the higher-order thinking and development through manifold active intellectual engagement aiming at meaning construction through pattern recognition and concept association. It includes inquiry, critical thinking, creative thinking, problem-solving, and metacognitive skills. It is a theory with a long academic record that can accommodate the demand for excellence in teaching and learning at all levels of education. Its achievement is verified through knowledge application in authentic contexts. 

  • pedagogy
  • instructional design
  • teaching
  • deep learning
  • meaningful learning
  • significant learning
  • deeper learning
  1. Introduction

1. Introduction

Equitable quality education and life-long learning opportunities for all is one of the United Nation’s seventeen global goals for sustainable development [1]. These goals comprise a compass for all countries and citizens for peaceful, global development and transformation by 2030. Quality higher education provides graduates with a robust combination of durable competencies, theoretical knowledge and procedural skills [2]. Life-long learning is of growing importance for the reskilling and upskilling of the workforce in the era of the fourth industrial revolution [3]. In the context of the COVID-19 pandemic and the imposed social distancing measures, there is also an acute need to improve the quality of distance education by transforming emergency remote teaching into deep online e-learning [4].

Equitable quality education and life-long learning opportunities for all is one of the United Nation’s seventeen global goals for sustainable development [1]. These goals comprise a compass for all countries and citizens for peaceful, global development and transformation by 2030. Quality higher education provides graduates with a robust combination of durable competencies, theoretical knowledge and procedural skills [2]. Life-long learning is of growing importance for the reskilling and upskilling of the workforce in the era of the fourth industrial revolution [3]. In the context of the COVID-19 pandemic and the imposed social distancing measures, there is also an acute need to improve the quality of distance education by transforming emergency remote teaching into deep online e-learning [4].

2. Model and Influences

2.1. Deep Learning

Deep learning originates from the research on the mental processing strategies by Marton and Säljö in Sweden [5]. In a series of experiments, they examined students’ approaches to learning when prompted to reply to comprehension questions after reading a text. They discovered two distinct behaviors; some students strived to store isolated facts without any reflection (surface approach). Others processed them critically and attempted to connect the new information with existing knowledge (deep approach). A student, employing deep learning approaches directs her own learning, attempts to comprehend the learning content and procedure, and modify accordingly his/her beliefs, behavior and values [5]. On the opposite end of the spectrum, a learner with a surface approach is rather apathetic towards the studied domain, driven by exam pressure or stress and hence opts to rote facts memorization. Beyond these two orientations, there is evidence of another, superseding pragmatic dimension towards short-term performance dictated by course assessment requirements, namely a strategic approach to learning [6].
The differences between a deep and a surface approach to learning are illustrated in the following example: John and Melissa attend the obligatory, core course on fluid mechanics towards a degree of Mechanical Engineering. John has a strong interest in industrial engineering and does not see how this course can be of any use to him in the short or long run. Therefore, he skips or is rather inattentive in classes and study. He intends to perform the bare minimum possible to get a passable grade in the final exam. Melissa is fascinated by the course’s links to previous courses on mathematics as well as future applications in various fields. She takes notes during lectures, asks questions, and is driven to search and study additional material beyond the course’s textbook. John’s attitude is an example of a surface approach to learning while Melissa exhibits a deep or meaningful, in-depth approach to learning.
The same researchers went on to formulate a hierarchy of six conceptions of learning, phases that students experience during their study [7]. The lowest three conceptions consist of surface approaches to learning: quantitative knowledge accumulation, memorization and storing, fact acquisition for future utilization. The next three phases are typical of the deep learning approach: sense-making through abstraction, reconceptualizing reality interpretation, and finally holistic person growth [7].
In addition, there is an alternative view towards deep learning. More specifically, Ohlsson conceptualized deep learning as the ability to perform essential, non-monotonic, cognitive development and change [8]. Among others, he identified three categories of non-monotonic mental shift:
(i).
capability to produce new solutions to problems and reach creative insights,
(ii).
adaptation of cognitive competencies through repetitive experimentation, and
(iii).
shift in values and perceptions through critical thinking [8].
Deep learning happens through active student engagement and especially in meaningful construction activities [9]. Deep learning is associated with polymorphic thinking (i.e., creative, critical, reflective, and caring) [10] and problem-solving processes and capabilities [11]. The notion of in-depth learning should not be confused with deep learning computational processing techniques used for data analysis and representation in the field of artificial intelligence.

2.2. Meaningful Learning

Meaningful learning, according to Ausubel [12], should be the hallmark of formal higher education, which is achieved through sustained critical discourse. Meaningful learning construction is linked with teaching methods such as inquiry and problem solving resulting in the ability to identify and analyze the underlying structure and connect existing with new concepts [13][14]. Educators who intend to offer meaningful educational experiences to their students are invited to contemplate and design teaching and learning around the following attributes: active, constructive, intentional, authentic, cooperative, or relational [15][16].
  • Active: Learning is an active cognitive procedure where the student is the protagonist. This dimension signals the active participation of learners by interacting with content and the learning environment, and engaging with a subject matter so as to make a personal cognitive contribution.
  • Constructive: Learners are expected to construct continuously their own meaning by interpreting and reflecting on observed phenomena, content and the results of their actions.
  • Intentional: Learners are encouraged to exhibit individual ownership, agency, be self-directed, set goals consciously and commit emotionally.
  • Authentic: Meaningful learning requires tasks linked to an authentic experience or simulated, realistic context so that they become personally significant and transferable.
  • Cooperative/relational: Human learning is also a social process involving learners and teachers. Group collaboration and peer conversation occur naturally in knowledge-building communities. Additionally, engaged, passionate teachers contribute significantly to the emotional involvement of learners.
Meaningful learning depends primarily on course design linking theory and practice with strong experiences where both teachers and students feel free to express their positive or negative emotions [17].

2.3. Deep and Meaningful Learning

Deep learning and meaningful learning have structural similarities that signal high quality in education and thus are integrated into the term deep and meaningful learning (DML) [18].
  1. Model and Influences

3. Related Theories

DML overlaps with other relevant concepts and theoretical frameworks with similar epistemological underpinnings in literature. These are significant learning, transformative learning, generative learning, deeper learning, and transfer of learning.

3.1. Significant Learning

Significant learning generates durable knowledge that can be applied in authentic contexts. It is achieved through student-centered teaching experiences driving personal learner cognitive development [19]. Significant learning requires multilevel mental student engagement across several categories [20]. Fink [21] proposed a taxonomy of the following six critical categories that can be used to formulate intended learning outcomes for interactive learning experiences:
  • Foundational knowledge; remembering and understanding the fundamental concepts in the core of an educational program’s content.
  • Application; identifying, analyzing a problem and solving it by applying the basic knowledge or skills.
  • Integration; building conceptual connections between new and existing knowledge and experiences.
  • Human dimension; recording an insight in the social dimension in relation to the self and other.
  • Caring; an emotional shift in regarding their values, perceptions and interest towards the studied domain.
  • Learning how to learn: acquiring domain-specific self-regulation skills to pursue life-long learning.
Educators seeking to ensure significant learning are encouraged to design and plan various learning activities across all categories.

3.2. Transformative Learning

Mezirow’s transformative learning is a much researched and studied adult education theory based on the critical theory [22]. Critical theory takes a clear stance towards the progressive transformation and emancipation of persons and society as a whole. It strives to discover the underlying or served interests in studied situations. It notes for example that the selection of information and methods in curriculum design is an ideological action [23]. Transformative learning emphasizes personal development, the evolution of worldview and perspectives through critical discourse and rational thinking [24]. This path of attitude transformation includes several steps: quandaries to trigger self-reflection leading to realizations and new decisions, exploring new, better and valid choices and devising plans towards behavioral change, putting new resolutions and values into action [25]

3.3. Generative Learning

Generative learning is based on the constructivist premise that knowledge is constructed through active student agency and participation [26]. Wittrock’s generative learning model includes four main stages: motivation, learning strategy, generation, and knowledge creation. However, one essential element is that learners need to assume responsibility, control and direct their own learning. For example, deep learning is more probable when learners are prompted to produce their own replies in the form of a written text to address an open question rather than select one option in a close-format multiple-choice question [27]. Generative learning involves active sense-making activities [28].

3.4. Deeper Learning

Deeper learning advocates learning beyond rote, superficial fact accumulation. Deeper learning is associated with higher-order thinking skills and mastery of transversal skills [29]. Deeper learning has the potential to deliver desirable effects such as enhanced information recall, intrinsic incentives, lasting knowledge and a structured comprehension of the cardinal propositions of the conceptual and procedural phenomena under scrutiny [30]. It aims at the development of six core competencies: proficiency of core academic content; critical thinking and complex problem solving; cooperation; communication; life-long learning; academic mindset. To cultivate these competencies teaching strategies such as problem-based and project-based learning have been found effective [31]. Active, student-centered instructional approaches are recommended including authentic case studies, small group work, interdisciplinary projects, mentorships, open-ended exploration, knowledge application outside of the classroom boundaries, personalized learning according to individual needs [32].

3.5. Transfer of Learning

Educational transfer or the transfer of learning is the phenomenon where a learner has the capability to demonstrate competencies, knowledge, skills, and values, acquired from educational settings to novel, unprecedented situations, and ill-defined problems [33]. For transfer to take place, learning needs to be organized as an active and dynamic process that is influenced by learners’ motives [34]. Educational transfer is considered a top priority in continuous professional development and corporate training programs.

2.1. Deep Learning

4. Application

How could DML be facilitated in the context of formal education? DML frameworks conceptualize education quality as the cognitive, affective, and social skills activation [21][35]. DML success in physical and online contexts depends on every individual’s idiosyncratic attributes in terms of personalities, abilities, perceptions, and goals [14]. Hence DML on scale requires adaptation and differentiation to accommodate personalized needs. Education stakeholders need to orchestrate litanies of activities and experiences to foster deep learning approaches [36]. DML from the educator’s angle is a tough challenge as it entails the expenditure of extra energy for sophisticated planning, patience, mindfulness, and diligence [14]. Information and communication technology could support DML when the latter is used for teaching and learning strategies such as knowledge synthesis, discussion, articulation, cooperation, and reflection [13][15][37].
DML is even harder to achieve and maintain in online learning where learners’ dynamic emotional and motivational fluctuations are sometimes neglected [38]. For instance, curiosity, interest, and goal orientation are essential as they influence directly cognitive learning procedures [39]. Quality e-learning towards higher-order processes should be organized around learner-centered meaningful, demanding activities assisting students to build associations of new information with existing knowledge and experiences [40].
More specific, DML is influenced by factors of three types: learners’ individual traits (e.g., personality, skills, emotions, motivation), contextual (e.g., teaching methods, assessment, teacher, class), and perceived contextual factors (e.g., workload, usefulness, relevance) [38]. In the context of distance education, a systematic review has integrated fifteen influencing factors into a blended model for deep and meaningful e-learning in social virtual reality environments [41]. Factors are organized in three classes: in relation to the learner (e.g., perceptions, technical skills), the implemented instructional design according to teacher perceptions and beliefs (e.g., learning theory, environment, activities), and the used technology (e.g., access, usability), before and during learning.
Hence, the community of inquiry theory was formulated to promote DML in tertiary education [42]. Deriving from a social constructivist epistemology, its empirically supported premise is that effective distant educational experiences should combine three crucial components: teaching, cognitive, and social presence. Teaching presence comprises the responsibilities and actions of educators such as instructional design, direct instruction, and online facilitation. Cognitive and social presence relates to student behavior. Cognitive presence is “the extent to which the participants in any particular configuration of a community of inquiry are able to construct meaning through sustained communication” [35]. Social presence is achieved when learners communicate purposively and build collectively shared identities in an environment of trust.
Online learning features principally flexible, self-regulated study. Even when learning features synchronous virtual meetings, i.e., teacher-led tutorials or group work, learner isolation is an inherently inhibiting factor [37][43][44]. Active, challenging activities, cooperative problem-based tasks, and emotional empowerment are recommended to promote DML [45]. Additionally, overlooking the importance of internal student incentives in distance education leads to high course attrition rates [46]. When distance students cannot interact socially with their fellows they have a higher probability of abandoning a course [47]. This effect has been observed on a magnified scale in Massive Open Online Courses (MOOCs). Global enrollment in each MOOC rose to thousands and even hundreds of thousands but completion rates typically do not exceed ten percent [14][48].
Excessive coursework is one common, DML blocking mistake educators commit despite their benevolent intentions is. Too much work inevitably pushes students towards a surface approach to learning due to time pressure. Hence, reducing content is recommended so that learners have the time to reflect on the studied subject [18]. Another universal teacher recommendation towards DML is to allow students to confront their own misconceptions. Learners should be animated to demonstrate comparatively their constructed meaning and interpretations of the studied domain and debate with each other [18].
DML proposes an outcome or competency-based design approach in e-learning [49]. Research in distance education connects DML with active learning, peer communication, and collaboration [50] as well as high levels of teaching and social presence [14][51]. Meaningful e-learning relies on the quality rather than the quantity of meaningful online interactions of learners with content, instructors, and peers [52]. These interactions should be designed around realistic experiences necessitating complex knowledge construction tasks with ample cooperation and reflection opportunities [14][53][54]. Game-based and gamified interventions such as serious games in physical and online, virtual settings have produced supporting evidence of DML [55][56]. Distance courses designed with constructivist principles integrating community interactions, open-ended discussions, and team assignments into a flexible curriculum with fluid content achieve higher levels of learner satisfaction and deep learning [57].

Deep learning originates from the research on the mental processing strategies by Marton and Säljö in Sweden [5]. In a series of experiments, they examined students’ approaches to learning when prompted to reply to comprehension questions after reading a text. They discovered two distinct behaviors; some students strived to store isolated facts without any reflection (surface approach). Others processed them critically and attempted to connect the new information with existing knowledge (deep approach). A student, employing deep learning approaches directs her own learning, attempts to comprehend the learning content and procedure, and modify accordingly his/her beliefs, behavior and values [5]. On the opposite end of the spectrum, a learner with a surface approach is rather apathetic towards the studied domain, driven by exam pressure or stress and hence opts to rote facts memorization. Beyond these two orientations, there is evidence of another, superseding pragmatic dimension towards short-term performance dictated by course assessment requirements, namely a strategic approach to learning [6].

5. Evaluation

Summative student assessment in formal education serves one main purpose: to ascertain the degree to which course participants have achieved the intended learning outcomes. Its format, however, constitutes an indirect hint to students as what is deemed of the highest value to focus on and learn [58]. Hence, a course aiming at deep meaningful knowledge development should examine higher-order competencies. Proposed evaluation strategies include authentic, realistic performance tasks, self-evaluation, and peer assessment [59][60]. Suggested assessment methods to encourage deep learning approaches are catalytic assessment, concept maps, problem-based learning, and e-portfolios [18][61].
Catalytic assessment starts with a question that students have to tackle [61]. The quest to find the right answer triggers first individual exploration and then discourse, often in dyads or larger teams where students present and defend their choices. Catalytic assessment can be applied in large audiences in physical and online settings as demonstrated by the peer instruction method [62].
Although concepts maps are learning resources, their creation by students can be a form of assessment [63]. Concept maps demonstrate a person’s cognitive organization of comprehension of a topic. Building links, hierarchical structures, and branches among related concepts, processes, and categories allows the accurate representation of students’ mental models.
Problem-based learning is a learner-centered method that starts with a real, ill-defined problem [11]. In order to solve the problem, students have to take initiative and direct their own learning in multiple ways: analyze the situation, identify its components, study sources, collect evidence, formulate and test hypotheses, communicate with peers, argue and take decisions, experiment, and validate their beliefs and assumptions.
Learning portfolios are collections of nowadays mostly digital artifacts (e.g., essays, papers, projects, digital files, etc.) that students build gradually throughout the course 59. Portfolios, similarly to PBL, place the responsibility and initiative of learning to each learner. Moreover, they strengthen learners’ agency and relatedness with personally meaningful values and connections. E-portfolios have the additional advantage that they can be transferable to other digital platforms and visible to social networks and other outlets enabling a seamless transition from educational to professional roles and settings [64]. In this way, portfolios encourage students’ intrinsic goal orientation.

The differences between a deep and a surface approach to learning are illustrated in the following example: John and Melissa attend the obligatory, core course on fluid mechanics towards a degree of Mechanical Engineering. John has a strong interest in industrial engineering and does not see how this course can be of any use to him in the short or long run. Therefore, he skips or is rather inattentive in classes and study. He intends to perform the bare minimum possible to get a passable grade in the final exam. Melissa is fascinated by the course’s links to previous courses on mathematics as well as future applications in various fields. She takes notes during lectures, asks questions, and is driven to search and study additional material beyond the course’s textbook. John’s attitude is an example of a surface approach to learning while Melissa exhibits a deep or meaningful, in-depth approach to learning.

6. Research Instruments

In an attempt to describe and classify the level, depth, complexity and quality of student learning and understanding, Biggs and Collis formulated the Structure of the Observed Learning Outcome taxonomy (SOLO), a hierarchy of five stages for learning outcomes [65]. These categories are the following from lowest to highest order:
  • Prestructural: Unstructured, inappropriate work.
  • Unistructural: Appropriate presentation of one relevant subject aspect.
  • Multistructural: Appropriate presentation of several relevant but unconnected subject aspects.
  • Relational: Integration of several relevant subject aspects.
  • Extended Abstract: Creation of a coherent, holistic approach at a new abstraction level.
SOLO taxonomy distinguishes two phases in student learning, intended or recorded. In the lowest, quantitative phase (stages 1 to 3), learning is mainly superficial, additive. In the qualitative phase (stages 4 and 5), learning results in advanced, deeper understanding, the ability of application, reflective abstraction and transfer. SOLO categories have correspondences with the six levels of Bloom’s revised taxonomy (remembering, understanding, applying, analyzing, evaluating, creating) [66]. SOLO can be used by educators in the design and assessment stage of education: to formulate learning objectives, techniques, activities, evaluation methods and to assess students’ outcomes and performance [67].
DML can be researched both with qualitative and quantitative methods. A qualitative DML research approach is phenomenography [68]. It constitutes a new research paradigm aiming at interpreting differences in thought and experiences based on the descriptions of understanding [69].
Validated quantitative research instruments to measure subjectively DML include the Study Process Questionnaire SPQ [70], the Approaches and Study Skills Inventory for Students (ASSIST) [71], the Motivated Strategies for Learning Questionnaire (MSLQ) [72], and the Community of Inquiry framework survey [73].
SPQ and more specifically the Revised Two-Factor Study Process Questionnaire (R-SPQ-2F) is a questionnaire developed by Biggs that measures two factors, deep and surface study approach [70]. It consists of twenty items, e.g., “my aim is to pass the course while doing as little work as possible” (surface study approach), “I feel that virtually any topic can be highly interesting once I get into it” (deep study approach). Students’ replies are scored on a five-point scale from “this is never or very rarely true of me” to “this always or almost always true of me”. R-SPQ-2F can be combined with SOLO taxonomy to link student study strategies to learning outcomes [74].
ASSIST is a self-reporting questionnaire that reflects relative student preferences towards three studying approaches: deep, surface and strategic, stemming from the work of Entwistle and Ramsden [71]. It contains three sections with the main section being the Revised Approaches to Studying Inventory (RASI). RASI includes 52 items, e.g., “I tend to read very little beyond what is actually required to pass” (surface approach), “Before tackling a problem or assignment, I first try to work out what lies behind it” (deep approach), I organize my study time carefully to make the best use of it (strategic approach). Students are invited to mark their degree of (dis)agreement across a five-level Likert type scale: agree, agree somewhat, unsure, disagree somewhat, agree.
MSLQ is based on Pintrich’s socio-cognitive assumption on learning depending primarily on the dynamic and contextual interplay between cognitive learning strategies and motivation orientation [75]. MSLQ can be used to measure 15 different motivation and learning strategy scales that can be used collectively or separately, e.g., intrinsic and extrinsic goals, self-efficacy, critical thinking, self-regulation, management of resources [72]. It contains 81 statements students assess ranging from 1 (not at all true of me) to 7 (very true of me), e.g., “I’m confident I can learn the basic concepts taught in this course”, “When studying for this course, I often try to explain the material to a classmate or friend”.
The Community of Inquiry framework survey was developed to measure the three primary scales of the studied model: cognitive, teaching, and social presence [73]. It comprises 34 items—statements such as “The instructor clearly communicated important course goals” and “Course activities piqued my curiosity”. Respondents are scored from 0 (strongly disagree) to 4 (strongly agree).

The same researchers went on to formulate a hierarchy of six conceptions of learning, phases that students experience during their study [7]. The lowest three conceptions consist of surface approaches to learning: quantitative knowledge accumulation, memorization and storing, fact acquisition for future utilization. The next three phases are typical of the deep learning approach: sense-making through abstraction, reconceptualizing reality interpretation, and finally holistic person growth [7].

In addition, there is an alternative view towards deep learning. More specifically, Ohlsson conceptualized deep learning as the ability to perform essential, non-monotonic, cognitive development and change [8]. Among others, he identified three categories of non-monotonic mental shift:

  • capability to produce new solutions to problems and reach creative insights,
  • adaptation of cognitive competencies through repetitive experimentation, and
  • shift in values and perceptions through critical thinking [8].

Deep learning happens through active student engagement and especially in meaningful construction activities [9]. Deep learning is associated with polymorphic thinking (i.e., creative, critical, reflective, and caring) [10] and problem-solving processes and capabilities [11]. The notion of in-depth learning should not be confused with deep learning computational processing techniques used for data analysis and representation in the field of artificial intelligence.

2.2. Meaningful Learning

Meaningful learning, according to Ausubel [12], should be the hallmark of formal higher education, which is achieved through sustained critical discourse. Meaningful learning construction is linked with teaching methods such as inquiry and problem solving resulting in the ability to identify and analyze the underlying structure and connect existing with new concepts [13,14]. Educators who intend to offer meaningful educational experiences to their students are invited to contemplate and design teaching and learning around the following attributes: active, constructive, intentional, authentic, cooperative, or relational [15,16].

  • Active: Learning is an active cognitive procedure where the student is the protagonist. This dimension signals the active participation of learners by interacting with content and the learning environment, and engaging with a subject matter so as to make a personal cognitive contribution.
  • Constructive: Learners are expected to construct continuously their own meaning by interpreting and reflecting on observed phenomena, content and the results of their actions.
  • Intentional: Learners are encouraged to exhibit individual ownership, agency, be self-directed, set goals consciously and commit emotionally.
  • Authentic: Meaningful learning requires tasks linked to an authentic experience or simulated, realistic context so that they become personally significant and transferable.
  • Cooperative/relational: Human learning is also a social process involving learners and teachers. Group collaboration and peer conversation occur naturally in knowledge-building communities. Additionally, engaged, passionate teachers contribute significantly to the emotional involvement of learners.

Meaningful learning depends primarily on course design linking theory and practice with strong experiences where both teachers and students feel free to express their positive or negative emotions [17].

2.3. Deep and Meaningful Learning

Deep learning and meaningful learning have structural similarities that signal high quality in education and thus are integrated into the term deep and meaningful learning (DML) [18].

  1. Related Theories

DML overlaps with other relevant concepts and theoretical frameworks with similar epistemological underpinnings in literature. These are significant learning, transformative learning, generative learning, deeper learning, and transfer of learning.

3.1. Significant Learning

Significant learning generates durable knowledge that can be applied in authentic contexts. It is achieved through student-centered teaching experiences driving personal learner cognitive development [19]. Significant learning requires multilevel mental student engagement across several categories [20]. Fink [21] proposed a taxonomy of the following six critical categories that can be used to formulate intended learning outcomes for interactive learning experiences:

  1. Foundational knowledge; remembering and understanding the fundamental concepts in the core of an educational program’s content.
  2. Application; identifying, analyzing a problem and solving it by applying the basic knowledge or skills.
  3. Integration; building conceptual connections between new and existing knowledge and experiences.
  4. Human dimension; recording an insight in the social dimension in relation to the self and other.
  5. Caring; an emotional shift in regarding their values, perceptions and interest towards the studied domain.
  6. Learning how to learn: acquiring domain-specific self-regulation skills to pursue life-long learning.

Educators seeking to ensure significant learning are encouraged to design and plan various learning activities across all categories.

3.2. Transformative Learning

Mezirow’s transformative learning is a much researched and studied adult education theory based on the critical theory [22]. Critical theory takes a clear stance towards the progressive transformation and emancipation of persons and society as a whole. It strives to discover the underlying or served interests in studied situations. It notes for example that the selection of information and methods in curriculum design is an ideological action [23]. Transformative learning emphasizes personal development, the evolution of worldview and perspectives through critical discourse and rational thinking [24]. This path of attitude transformation includes several steps: quandaries to trigger self-reflection leading to realizations and new decisions, exploring new, better and valid choices and devising plans towards behavioral change, putting new resolutions and values into action [25].

3.3. Generative Learning

Generative learning is based on the constructivist premise that knowledge is constructed through active student agency and participation [26]. Wittrock’s generative learning model includes four main stages: motivation, learning strategy, generation, and knowledge creation. However, one essential element is that learners need to assume responsibility, control and direct their own learning. For example, deep learning is more probable when learners are prompted to produce their own replies in the form of a written text to address an open question rather than select one option in a close-format multiple-choice question [27]. Generative learning involves active sense-making activities [28].

3.4. Deeper Learning

Deeper learning advocates learning beyond rote, superficial fact accumulation. Deeper learning is associated with higher-order thinking skills and mastery of transversal skills [29]. Deeper learning has the potential to deliver desirable effects such as enhanced information recall, intrinsic incentives, lasting knowledge and a structured comprehension of the cardinal propositions of the conceptual and procedural phenomena under scrutiny [30]. It aims at the development of six core competencies: proficiency of core academic content; critical thinking and complex problem solving; cooperation; communication; life-long learning; academic mindset. To cultivate these competencies teaching strategies such as problem-based and project-based learning have been found effective [31]. Active, student-centered instructional approaches are recommended including authentic case studies, small group work, interdisciplinary projects, mentorships, open-ended exploration, knowledge application outside of the classroom boundaries, personalized learning according to individual needs [32].

3.5. Transfer of Learning

Educational transfer or the transfer of learning is the phenomenon where a learner has the capability to demonstrate competencies, knowledge, skills, and values, acquired from educational settings to novel, unprecedented situations, and ill-defined problems [33]. For transfer to take place, learning needs to be organized as an active and dynamic process that is influenced by learners’ motives [34]. Educational transfer is considered a top priority in continuous professional development and corporate training programs.

  1. Application

How could DML be facilitated in the context of formal education? DML frameworks conceptualize education quality as the cognitive, affective, and social skills activation [21,35]. DML success in physical and online contexts depends on every individual’s idiosyncratic attributes in terms of personalities, abilities, perceptions, and goals [14]. Hence DML on scale requires adaptation and differentiation to accommodate personalized needs. Education stakeholders need to orchestrate litanies of activities and experiences to foster deep learning approaches [36]. DML from the educator’s angle is a tough challenge as it entails the expenditure of extra energy for sophisticated planning, patience, mindfulness, and diligence [14]. Information and communication technology could support DML when the latter is used for teaching and learning strategies such as knowledge synthesis, discussion, articulation, cooperation, and reflection [13,15,37].

DML is even harder to achieve and maintain in online learning where learners’ dynamic emotional and motivational fluctuations are sometimes neglected [38]. For instance, curiosity, interest, and goal orientation are essential as they influence directly cognitive learning procedures [39]. Quality e-learning towards higher-order processes should be organized around learner-centered meaningful, demanding activities assisting students to build associations of new information with existing knowledge and experiences [40].

More specific, DML is influenced by factors of three types: learners’ individual traits (e.g., personality, skills, emotions, motivation), contextual (e.g., teaching methods, assessment, teacher, class), and perceived contextual factors (e.g., workload, usefulness, relevance) [38]. In the context of distance education, a systematic review has integrated fifteen influencing factors into a blended model for deep and meaningful e-learning in social virtual reality environments [41]. Factors are organized in three classes: in relation to the learner (e.g., perceptions, technical skills), the implemented instructional design according to teacher perceptions and beliefs (e.g., learning theory, environment, activities), and the used technology (e.g., access, usability), before and during learning.

Hence, the community of inquiry theory was formulated to promote DML in tertiary education [42]. Deriving from a social constructivist epistemology, its empirically supported premise is that effective distant educational experiences should combine three crucial components: teaching, cognitive, and social presence. Teaching presence comprises the responsibilities and actions of educators such as instructional design, direct instruction, and online facilitation. Cognitive and social presence relates to student behavior. Cognitive presence is “the extent to which the participants in any particular configuration of a community of inquiry are able to construct meaning through sustained communication” [35]. Social presence is achieved when learners communicate purposively and build collectively shared identities in an environment of trust.

Online learning features principally flexible, self-regulated study. Even when learning features synchronous virtual meetings, i.e., teacher-led tutorials or group work, learner isolation is an inherently inhibiting factor [37,43,44]. Active, challenging activities, cooperative problem-based tasks, and emotional empowerment are recommended to promote DML [45]. Additionally, overlooking the importance of internal student incentives in distance education leads to high course attrition rates [46]. When distance students cannot interact socially with their fellows they have a higher probability of abandoning a course [47]. This effect has been observed on a magnified scale in Massive Open Online Courses (MOOCs). Global enrollment in each MOOC rose to thousands and even hundreds of thousands but completion rates typically do not exceed ten percent [14,48].

Excessive coursework is one common, DML blocking mistake educators commit despite their benevolent intentions is. Too much work inevitably pushes students towards a surface approach to learning due to time pressure. Hence, reducing content is recommended so that learners have the time to reflect on the studied subject [18]. Another universal teacher recommendation towards DML is to allow students to confront their own misconceptions. Learners should be animated to demonstrate comparatively their constructed meaning and interpretations of the studied domain and debate with each other [18].

DML proposes an outcome or competency-based design approach in e-learning [49]. Research in distance education connects DML with active learning, peer communication, and collaboration [50] as well as high levels of teaching and social presence [14,51]. Meaningful e-learning relies on the quality rather than the quantity of meaningful online interactions of learners with content, instructors, and peers [52]. These interactions should be designed around realistic experiences necessitating complex knowledge construction tasks with ample cooperation and reflection opportunities [14,53,54]. Game-based and gamified interventions such as serious games in physical and online, virtual settings have produced supporting evidence of DML [55,56]. Distance courses designed with constructivist principles integrating community interactions, open-ended discussions, and team assignments into a flexible curriculum with fluid content achieve higher levels of learner satisfaction and deep learning [57].

  1. Evaluation

Summative student assessment in formal education serves one main purpose: to ascertain the degree to which course participants have achieved the intended learning outcomes. Its format, however, constitutes an indirect hint to students as what is deemed of the highest value to focus on and learn [58]. Hence, a course aiming at deep meaningful knowledge development should examine higher-order competencies. Proposed evaluation strategies include authentic, realistic performance tasks, self-evaluation, and peer assessment [59,60]. Suggested assessment methods to encourage deep learning approaches are catalytic assessment, concept maps, problem-based learning, and e-portfolios [18,61].

Catalytic assessment starts with a question that students have to tackle [61]. The quest to find the right answer triggers first individual exploration and then discourse, often in dyads or larger teams where students present and defend their choices. Catalytic assessment can be applied in large audiences in physical and online settings as demonstrated by the peer instruction method [62].

Although concepts maps are learning resources, their creation by students can be a form of assessment [63]. Concept maps demonstrate a person’s cognitive organization of comprehension of a topic. Building links, hierarchical structures, and branches among related concepts, processes, and categories allows the accurate representation of students’ mental models.

Problem-based learning is a learner-centered method that starts with a real, ill-defined problem [11]. In order to solve the problem, students have to take initiative and direct their own learning in multiple ways: analyze the situation, identify its components, study sources, collect evidence, formulate and test hypotheses, communicate with peers, argue and take decisions, experiment, and validate their beliefs and assumptions.

Learning portfolios are collections of nowadays mostly digital artifacts (e.g., essays, papers, projects, digital files, etc.) that students build gradually throughout the course [59]. Portfolios, similarly to PBL, place the responsibility and initiative of learning to each learner. Moreover, they strengthen learners’ agency and relatedness with personally meaningful values and connections. E-portfolios have the additional advantage that they can be transferable to other digital platforms and visible to social networks and other outlets enabling a seamless transition from educational to professional roles and settings [64]. In this way, portfolios encourage students’ intrinsic goal orientation.

  1. Research Instruments

In an attempt to describe and classify the level, depth, complexity and quality of student learning and understanding, Biggs and Collis formulated the Structure of the Observed Learning Outcome taxonomy (SOLO), a hierarchy of five stages for learning outcomes [65]. These categories are the following from lowest to highest order:

  1. Prestructural: Unstructured, inappropriate work.
  2. Unistructural: Appropriate presentation of one relevant subject aspect.
  3. Multistructural: Appropriate presentation of several relevant but unconnected subject aspects.
  4. Relational: Integration of several relevant subject aspects.
  5. Extended Abstract: Creation of a coherent, holistic approach at a new abstraction level.

SOLO taxonomy distinguishes two phases in student learning, intended or recorded. In the lowest, quantitative phase (stages 1 to 3), learning is mainly superficial, additive. In the qualitative phase (stages 4 and 5), learning results in advanced, deeper understanding, the ability of application, reflective abstraction and transfer. SOLO categories have correspondences with the six levels of Bloom’s revised taxonomy (remembering, understanding, applying, analyzing, evaluating, creating) [66]. SOLO can be used by educators in the design and assessment stage of education: to formulate learning objectives, techniques, activities, evaluation methods and to assess students’ outcomes and performance [67].

DML can be researched both with qualitative and quantitative methods. A qualitative DML research approach is phenomenography [68]. It constitutes a new research paradigm aiming at interpreting differences in thought and experiences based on the descriptions of understanding [69].

Validated quantitative research instruments to measure subjectively DML include the Study Process Questionnaire SPQ [70], the Approaches and Study Skills Inventory for Students (ASSIST) [71], the Motivated Strategies for Learning Questionnaire (MSLQ) [72], and the Community of Inquiry framework survey [73].

SPQ and more specifically the Revised Two-Factor Study Process Questionnaire (R-SPQ-2F) is a questionnaire developed by Biggs that measures two factors, deep and surface study approach [70]. It consists of twenty items, e.g., “my aim is to pass the course while doing as little work as possible” (surface study approach), “I feel that virtually any topic can be highly interesting once I get into it” (deep study approach). Students' replies are scored on a five-point scale from “this is never or very rarely true of me” to “this always or almost always true of me”. R-SPQ-2F can be combined with SOLO taxonomy to link student study strategies to learning outcomes [74].

ASSIST is a self-reporting questionnaire that reflects relative student preferences towards three studying approaches: deep, surface and strategic, stemming from the work of Entwistle and Ramsden [71]. It contains three sections with the main section being the Revised Approaches to Studying Inventory (RASI). RASI includes 52 items, e.g., “I tend to read very little beyond what is actually required to pass” (surface approach), “Before tackling a problem or assignment, I first try to work out what lies behind it” (deep approach), I organize my study time carefully to make the best use of it (strategic approach). Students are invited to mark their degree of (dis)agreement across a five-level Likert type scale: agree, agree somewhat, unsure, disagree somewhat, agree.

MSLQ is based on Pintrich’s socio-cognitive assumption on learning depending primarily on the dynamic and contextual interplay between cognitive learning strategies and motivation orientation [75]. MSLQ can be used to measure 15 different motivation and learning strategy scales that can be used collectively or separately, e.g., intrinsic and extrinsic goals, self-efficacy, critical thinking, self-regulation, management of resources [72]. It contains 81 statements students assess ranging from 1 (not at all true of me) to 7 (very true of me), e.g., “I’m confident I can learn the basic concepts taught in this course”, “When studying for this course, I often try to explain the material to a classmate or friend”.

The Community of Inquiry framework survey was developed to measure the three primary scales of the studied model: cognitive, teaching, and social presence [73]. It comprises 34 items—statements such as “The instructor clearly communicated important course goals” and “Course activities piqued my curiosity”. Respondents are scored from 0 (strongly disagree) to 4 (strongly agree).

  1. Conclusions and Prospects

Life-long learning in the context of an information-centered society through continuous professional development is ubiquitous [76]. The quality of life-long learning is vital for the effectiveness of upskilling and reskilling professional development initiatives. Learning interventions and educational programs of high quality lead to DML. Future research lines could investigate the intersection of DML and behavioral change in blended and distance education with emerging technologies such as extended, cross, augmented, mixed, virtual reality as well as digital games [77], big data and learning analytics [78]. In a macroscopic view, DML is not an end, it is the beginning of passionate engagements of students with domains of knowledge fueled by inspiration through inquiry and experimentation leading to creativity, polymorphic innovation and solutions to pressing problems.

7. Conclusions and Prospects

Life-long learning in the context of an information-centered society through continuous professional development is ubiquitous [76]. The quality of life-long learning is vital for the effectiveness of upskilling and reskilling professional development initiatives. Learning interventions and educational programs of high quality lead to DML. Future research lines could investigate the intersection of DML and behavioral change in blended and distance education with emerging technologies such as extended, cross, augmented, mixed, virtual reality as well as digital games [77], big data and learning analytics [78]. In a macroscopic view, DML is not an end, it is the beginning of passionate engagements of students with domains of knowledge fueled by inspiration through inquiry and experimentation leading to creativity, polymorphic innovation and solutions to pressing problems.

References

  1. United Nations General Assembly. Transforming Our World: The 2030 Agenda for Sustainable Development; UN: New York, NY, USA, 2015.
  2. Greiff, S.; Wüstenberg, S.; Csapó, B.; Demetriou, A.; Hautamäki, J.; Graesser, A.C.; Martin, R. Domain-general problem solving skills and education in the 21st century. Educ. Res. Rev. 2014, 13, 74–83.
  3. Gleason, N.W. Higher Education in the Era of the Fourth Industrial Revolution; Springer Nature: Berlin/Heidelberg, Germany, 2018; ISBN 9789811301940.
  4. Schultz, R.B.; DeMers, M.N. Transitioning from Emergency Remote Learning to Deep Online Learning Experiences in Geography Education. J. Geog. 2020, 119, 142–146.
  5. Marton, F.; Säljö, R. On Qualitative Differences in Learning—II Outcome as a Function of the Learner’s Conception of the Task. Br. J. Educ. Psychol. 1976, 46, 115–127.
  6. Miller, C.M.L.; Parlett, M.R. Up to the Mark: A Study of the Examination Game. In Research into Higher Education Monographs; Society for Research into Higher Education: Guildford, UK, 1974; ISBN 9780900868375.
  7. Marton, F.; Säljö, R. Approaches to Learning. In The Experience of Learning; Marton, F., Hounsell, D., Entwistle, N., Eds.; Scottish Academic Press: Edinburgh, UK, 1997; pp. 39–58.
  8. Ohlsson, S. Deep Learning: How the Mind Overrides Experience; Cambridge University Press: Cambridge, UK, 2011; ISBN 9781139496759.
  9. Hay, D.B.; Kehoe, C.; Miquel, M.E.; Hatzipanagos, S.; Kinchin, I.M.; Keevil, S.F.; Lygo-Baker, S. Measuring the quality of e-learning. Br. J. Educ. Technol. 2008, 39, 1037–1056.
  10. Valtanen, J.; Berki, E.; Kampylis, P.; Theodorakopoulou, M. Manifold Thinking and Distributed Problem-Based Learning: Is There Potential For ICT Support? In Proceedings of the e-Learning’08 Conference, Las Vegas, NV, USA, 14–17 July 2008; Volume I, pp. 145–152.
  11. Dolmans, D.H.J.M.; Loyens, S.M.M.; Marcq, H.; Gijbels, D. Deep and surface learning in problem-based learning: A review of the literature. Adv. Heal. Sci. Educ. 2016, 21, 1087–1112.
  12. Ausubel, D.P. In Defense of Verbal Learning. Educ. Theory 1961, 11, 15–25.
  13. Jonassen, D.H. Learning to Solve Problems with Technology: A Constructivist Perspective, 2nd ed.; Merrill: Upper Saddle River, NJ, USA, 2003; ISBN 9780130484031.
  14. Mystakidis, S.; Berki, E.; Valtanen, J.-P. The Patras Blended Strategy Model for Deep and Meaningful Learning in Quality Life-Long Distance Education. Electron. J. e-Learning 2019, 17, 66–78.
  15. Howland, J.L.; Jonassen, D.H.; Marra, R.M. Meaningful Learning with Technology, 4th ed.; Pearson: London, UK, 2011; ISBN 9780132565585.
  16. Mystakidis, S. Motivation Enhanced Deep and Meaningful Learning with Social Virtual Reality; University of Jyväskylä: Jyväskylän yliopisto, Finland, 2019.
  17. Kostiainen, E.; Ukskoski, T.; Ruohotie-Lyhty, M.; Kauppinen, M.; Kainulainen, J.; Mäkinen, T. Meaningful learning in teacher education. Teach. Teach. Educ. 2018, 71, 66–77.
  18. Rourke, L.; Kanuka, H. Learning in Communities of Inquiry: A Review of the Literature. J. Distance Educ. 2009, 23, 19–48.
  19. Rogers, C. Client-Centered Therapy; Houghton-Mifflin: Boston, MA, USA, 1951.
  20. Delotell, P.J.; Millam, L.A.; Reinhardt, M.M. The Use of Deep Learning Strategies in Online Business Courses to Impact Student Retention. Am. J. Bus. Educ. 2010, 3, 49–56.
  21. Fink, L.D. Creating Significant Learning Experiences: An Integrated Approach to Designing College Courses; Jossey-Bass: San Francisco, CA, USA, 2003.
  22. Mezirow, J. Transformative Learning as Discourse. J. Transform. Educ. 2003, 1, 58–63.
  23. Cohen, L.; Manion, L.; Morrison, K. Research Methods in Education, 7th ed.; Taylor and Francis: London, UK, 2013.
  24. Christie, M.; Carey, M.; Robertson, A.; Grainger, P. Putting transformative learning theory into practice. Aust. J. Adult Learn. 2015, 55, 10–30.
  25. Illeris, K. Transformative Learning in the Perspective of a Comprehensive Learning Theory. J. Transform. Educ. 2004, 2, 79–89.
  26. Wittrock, M.C. Learning as a generative process. Educ. Psychol. 1974, 11, 87–95.
  27. Slamecka, N.J.; Graf, P. The generation effect: Delineation of a phenomenon. J. Exp. Psychol. Hum. Learn. Mem. 1978, 4, 592–604.
  28. Fiorella, L.; Mayer, R.E. Eight Ways to Promote Generative Learning. Educ. Psychol. Rev. 2016, 28, 717–741.
  29. Martinez, M.; McGrath, D. Deeper Learning: How Eight Innovative Public Schools Are Transforming Education in the Twenty-First Century; EBL-Schweitzer; New Press: New York, NY, USA, 2014; ISBN 9781595589941.
  30. Dede, C.; Grotzer, T.A.; Kamarainen, A.; Metcalf, S. EcoXPT: Designing for Deeper Learning through Experimentation in an Immersive Virtual Ecosystem. J. Educ. Technol. Soc. 2017, 20, 166–178.
  31. Sergis, S.; Sampson, D.G. Teaching and Learning Analytics to Support Teacher Inquiry: A Systematic Literature Review. In Learning Analytics: Fundaments, Applications, and Trends; Springer Nature: Cham, Switzerland, 2017; pp. 25–63.
  32. Dede, C. The Role of Technology in Deeper Learning; Jobs for the Future: New York, NY, USA, 2014.
  33. Ellis, H.C. The Transfer of Learning; Macmillan: Oxford, UK, 1965.
  34. Pugh, K.; Bergin, D. Motivational influences on transfer. Educ. Psychol. 2006, 41, 147–160.
  35. Garrison, D.R.; Anderson, T.; Archer, W. Critical Inquiry in a Text-Based Environment: Computer Conferencing in Higher Education. Internet High. Educ. 1999, 2, 87–105.
  36. Entwistle, N.; Peterson, J.; Elizabeth, R. Promoting deep learning through teaching and assessment: Conceptual frameworks and educational contexts. In Proceedings of the Teaching and Learning Research Programme (TLRP) Conference, Leicester, UK, 9–10 November 2000; pp. 9–20.
  37. Koszalka, T.A.; Pavlov, Y.; Wu, Y. The informed use of pre-work activities in collaborative asynchronous online discussions: The exploration of idea exchange, content focus, and deep learning. Comput. Educ. 2021, 161, 104067.
  38. Baeten, M.; Kyndt, E.; Struyven, K.; Dochy, F. Using student-centred learning environments to stimulate deep approaches to learning: Factors encouraging or discouraging their effectiveness. Educ. Res. Rev. 2010, 5, 243–260.
  39. Schiefele, U. Interest, Learning, and Motivation. Educ. Psychol. 1991, 26, 299–323.
  40. Bonk, C.J.; Reynolds, T.H. Learner-centered Web instruction for higher-order thinking, teamwork, and apprenticeship. In Web-Based Instruction; Khan, B.H., Ed.; Educational Technology Publications: Englewood Cliffs, NJ, USA, 1997; pp. 167–178.
  41. Mystakidis, S.; Berki, E.; Valtanen, J.-P. Deep and Meaningful E-Learning with Social Virtual Reality Environments in Higher Education: A Systematic Literature Review. Appl. Sci. 2021, 11, 2412.
  42. Garrison, D.R.; Anderson, T.; Archer, W. The first decade of the community of inquiry framework: A retrospective. Internet High. Educ. 2010, 13, 5–9.
  43. Paulus, T.; Scherff, L. Can Anyone Offer any Words of Encouragement? Online Dialogue as a Support Mechanism for Preservice Teachers. J. Technol. Teach. Educ. 2008, 16, 113–136.
  44. Mystakidis, S.; Berki, E.; Valtanen, J.-P.; Amanatides, E. Towards a Blended Strategy for Quality Distance Education Life-Long Learning Courses—The Patras Model. In Proceedings of the 17th European Conference on e-Learning (ECEL), Athens, Greece, 1–2 November 2018; pp. 408–416.
  45. Hacker, D.J.; Niederhauser, D.S. Promoting deep and durable learning in the online classroom. New Dir. Teach. Learn. 2000, 84, 53–63.
  46. Tyler-Smith, K. Early attrition among first time eLearners: A review of factors that contribute to drop-out, withdrawal and non-completion rates of adult learners undertaking eLearning programmes. J. Online Learn. Teach. 2006, 2, 73–85.
  47. Willging, P.A.; Johnson, S.D. Factors that Influence Students’ Decision to Dropout of Online Courses. J. Asynchronous Learn. Networks 2009, 13, 115–127.
  48. Jordan, K. Massive open online course completion rates revisited: Assessment, length and attrition. Int. Rev. Res. Open Distrib. Learn. 2015, 16, 341–358.
  49. Guàrdia, L.; Maina, M.; Sangrà, A. MOOC Design Principles. A Pedagogical Approach from the Learner’s Perspective. eLearning Pap. 2013, 33, 1–6.
  50. Morin, D.; Thomas, J.D.E.; Raafat, G.S. Deep Learning and Virtual Environment. Int. J. Psychol. Behav. Sci. 2012, 6, 31–63.
  51. Bangert, A. The influence of social presence and teaching presence on the quality of online critical inquiry. J. Comput. High. Educ. 2008, 20, 34–61.
  52. Yoon, S. In search of meaningful online learning experiences. New Dir. Adult Contin. Educ. 2003, 19–30.
  53. Woo, Y.; Reeves, T.C. Meaningful interaction in web-based learning: A social constructivist interpretation. Internet High. Educ. 2007, 10, 15–25.
  54. Garrison, D.R.; Cleveland-Innes, M. Facilitating Cognitive Presence in Online Learning: Interaction Is Not Enough. Am. J. Distance Educ. 2005, 19, 133–148.
  55. Mystakidis, S.; Cachafeiro, E.; Hatzilygeroudis, I. Enter the Serious E-scape Room: A Cost-Effective Serious Game Model for Deep and Meaningful E-learning. In Proceedings of the 2019 10th International Conference on Information, Intelligence, Systems and Applications (IISA), Patras, Greence, 15–17 July 2019; pp. 1–6.
  56. Pellas, N.; Mystakidis, S.; Christopoulos, A. A Systematic Literature Review on the User Experience Design for Game-Based Interventions via 3D Virtual Worlds in K-12 Education. Multimodal Technol. Interact. 2021, 5, 28.
  57. Ke, F.; Xie, K. Toward deep learning for adult students in online courses. Internet High. Educ. 2009, 12, 136–145.
  58. Biggs, J. What the Student Does: Teaching for enhanced learning. High. Educ. Res. Dev. 1999, 18, 57–75.
  59. Gikandi, J.W.; Morrow, D.; Davis, N.E. Online formative assessment in higher education: A review of the literature. Comput. Educ. 2011, 57, 2333–2351.
  60. Nieminen, J.H.; Asikainen, H.; Rämö, J. Promoting deep approach to learning and self-efficacy by changing the purpose of self-assessment: A comparison of summative and formative models. Stud. High. Educ. 2019, 1–16.
  61. Draper, S.W. Catalytic assessment: Understanding how MCQs and EVS can foster deep learning. Br. J. Educ. Technol. 2009, 40, 285–293.
  62. Crouch, C.H.; Mazur, E. Peer Instruction: Ten years of experience and results. Am. J. Phys. 2001, 69, 970–977.
  63. Novak, J.D.; Ridley, D.R. Assessing Student Learning in Light of How Students Learn. In AAHE Assessment Forum; American Association for Higher Education: Washington, DC, USA, 1988.
  64. Gibson, D.; Ostashewski, N.; Flintoff, K.; Grant, S.; Knight, E. Digital badges in education. Educ. Inf. Technol. 2015, 20, 403–410.
  65. Biggs, J.B.; Collis, K.F. Evaluating the Quality of Learning: The SOLO Taxonomy; Elsevier: Amsterdam, The Netherlands, 1982; ISBN 0120975505.
  66. Anderson, L.W.; Krathwohl, D.R.; Airasian, P.W.; Cruikshank, K.A.; Mayer, R.E.; Pintrich, P.R.; Raths, J.; Wittrock, M.C. A Taxonomy for Learning, Teaching, and Assessing: A Revision of Bloom’s Taxonomy of Educational Objectives, Abridged Edition; Pearson: London, UK, 2000; ISBN 080131903X.
  67. Leiva-Brondo, M.; Cebolla-Cornejo, J.; Peiró, R.; Andrés-Colás, N.; Esteras, C.; Ferriol, M.; Merle, H.; Díez, M.J.; Pérez-de-Castro, A. Study Approaches of Life Science Students Using the Revised Two-Factor Study Process Questionnaire (R-SPQ-2F). Educ. Sci. 2020, 10, 173.
  68. Marton, F. Phenomenography—Describing conceptions of the world around us. Instr. Sci. 1981, 10, 177–200.
  69. Marton, F. Phenomenography—A research approach to investigating different understandings of reality. J. Thought 1986, 21, 28–49.
  70. Biggs, J.; Kember, D.; Leung, D.Y.P. The revised two-factor Study Process Questionnaire: R-SPQ-2F. Br. J. Educ. Psychol. 2001, 71, 133–149.
  71. Entwistle, N.J.; McCune, V.; Tait, H. The Approaches and Study Skills Inventory for Students (ASSIST); Centre for Research on Learning and Instruction, University of Edinburgh: Edinburgh, UK, 1997.
  72. Pintrich, P.R.; Smith, D.A.F.; Garcia, T.; Mckeachie, W.J. Reliability and Predictive Validity of the Motivated Strategies for Learning Questionnaire (Mslq). Educ. Psychol. Meas. 1993, 53, 801–813.
  73. Arbaugh, J.B.; Cleveland-Innes, M.; Diaz, S.R.; Garrison, D.R.; Ice, P.; Richardson, J.C.; Swan, K.P. Developing a community of inquiry instrument: Testing a measure of the Community of Inquiry framework using a multi-institutional sample. Internet High. Educ. 2008, 11, 133–136.
  74. Rossum, E.J.; Schenk, S.M. The Relationship between Learning Conception, Study Strategy and Learning Outcome. Br. J. Educ. Psychol. 1984, 54, 73–83.
  75. Duncan, T.G.; McKeachie, W.J. The Making of the Motivated Strategies for Learning Questionnaire. Educ. Psychol. 2005, 40, 117–128.
  76. Bragg, L.; Walsh, C.; Heyeres, M. Successful design and delivery of online professional development for teachers: A systematic review of the literature. Comput. Educ. 2021, 104158.
  77. Grande-de-Prado, M.; García-Martín, S.; Baelo, R.; Abella-García, V. Edu-Escape Rooms. Encyclopedia 2021, 1, 4.
  78. Christopoulos, A.; Mystakidis, S.; Pellas, N.; Laakso, M.-J. ARLEAN: An Augmented Reality Learning Analytics Ethical Framework. Computers 2021, 10, 92.
More
Video Production Service