1. Introduction

The impact of the Internet Age has prompted a paradigm shift in education. Nowadays most of our everyday learning is characterized by drawing knowledge from a wide variety of electronic resources. Learners from different levels are required to search, collect, and understand information from digital external sources and construct a solution to solve a task. This shift has never been more noticeable than amidst the current coronavirus pandemic. In this context, it is important to remember that educational research has identified information problem solving (henceforth IPS) as a complex process that requires the unfolding of complex higher-order cognitive skills, e.g., ^[1][2][3].

Although it is undeniable that younger generations of students appear to master the skills needed to navigate online digital resources, educational research confirms that, without explicit instruction, students underuse or even lack the IPS skills to find correct and reliable online resources and construct knowledge from them ^[4][5][6][7]. Therefore, educational research sees the need to provide students with adequate IPS skills to learn from online and digital resources. Furthermore, ^[8] claim that IPS skills instruction is crucial to promote quality, equality, and sustainable education because it has been found that students’ performance in digital skills is initially associated with their socio-economic background, academic achievement and residence location.

Various theoretical models have been proposed to characterize the phases and the cognitive processes involved in IPS that are needed to transform the retrieved web information into knowledge ^[9]. However, these models describe the stages and cognitive competences involved in the process, but fail to show which the students’ specific activities are in each stage and how to best support them. As a consequence, educational institutions and teachers find it difficult to teach the key IPS skills that could help students take full advantage of the opportunities the Internet provides for learning and building knowledge autonomously from online digital resources and in finding a suitable place and time in the curriculum ^[3][5].

In recent years, research has been carried out to analyse the effectiveness of teaching IPS using the Internet, e.g. ^{[10][11][12][13]}. However, further research is still needed to tailor the existing IPS models to specific groups of students and in specific learning contexts ^[3][9] and, by so doing, promote quality and sustainable education for all students. 

This paper takes a first step towards supporting teachers in embedding IPS skills in educational curricula with the description of the design and empirical testing of instruction for IPS skills. Inspired by the four-component instructional design model - 4C/ID ^[14] for teaching complex skills, we have designed a long-term, embedded, whole-task IPS instruction to foster learning and meaning-making from digital sources, and investigated its effects on students’ task-performance.

1.1. Information problem solving

Information problem solving (IPS) is a complex cognitive process considered as an important 21^st century skill in combination with critical thinking ^[15]. ^[1][2][3] have defined a five-step approach to solving information problems based on a decomposition of the IPS process into constituent skills and subskills. This approach highlights the fact that during the implementation of all skills it is essential to activate regulation activities, such as orientation, monitoring, steering, and evaluating ^[16][17].

Figure 1. Five-step systematic approach to information problem solving, based on: ^[1][2][3].

Figure 1 shows this IPS model. Basically, it represents that, when students are confronted with an information problem or challenge. Considering that the resolution of the task as the solution to an information problem from online sources implies a complex cognitive process ^[12][16], in which secondary students face many challenges, it is essential for them to receive guidance and supervision through a well-designed educational intervention.

1.2.  Information problem solving instruction

It is often claimed that the IPS skills are underdeveloped or absent without explicit instruction, even among “digital natives” ^{[1][3][4][5][6][16]}. However, educational research shows that students can be instructed to define better the problem and the information needed, generate more relevant search queries, adopt more evaluation criteria, select higher quality resources and deeply processed and presented information to answer an informational problem ^[10][18].

Over the last decades much effort has been made to investigate efficient instructional approaches for IPS and incorporate effective support for guiding students’ activity in searching, retrieving, evaluating  and integrating  information from multiple web sources (e.g., ^{[3][4][10][11][12][13][16][19]}). However, despite the researchers’ efforts made so far, their attempts have proved insufficient and further research is still needed in order to face and shed light on how formal IPS skills training could be designed in order to have a positive impact on student’s learning.

Our study is built on the basis of the four-component instructional design (4C/ID, for short) model ^[14] to design, implement and empirically test an innovative IPS instruction in secondary education. The 4C/ID model advocates the design of four components:

1. Learning tasks are understood as authentic real-life tasks and their solution requires the integration and coordination of skills, knowledge and attitudes.
2. Both supportive information and guidance are needed to develop cognitive models and strategies in order to complete the learning task.
3. Procedural information has to be carefully designed by providing step-by-step instruction and explicit skills and procedures.
4. Part-task practice should be included to provide enough training for recurrent skills.

In the arena of IPS instruction these four components have been translated according to the following principles: whole-task, embedded, and long-term instruction.

1.2.1. Whole-task IPS instruction

Whole-task instruction proposes the resolution of ill-structured, authentic and complex real-life situations in which students have to perform all the steps of the IPS process, from beginning to end, and students can find different ways to solve the task. A whole-task instruction has proved more effective to teach IPS complex skills than part-task fragmented instruction ^[10][14]. Whole-task instruction offers the possibility to provide support for all the IPS skills and practise them as a whole process in which one skill relates to and impacts on the others. By contrast, instructional approaches that focus only on practising specific searching or evaluating skills, e.g. ^[20], offer students very few occasions to coordinate and integrate all of the five IPS skills ^[14], and also to transfer ^[21].

Regarding the support needed, research provides evidence that it is possible to build on whole-task support to improve students’ IPS skills in demanding learning and learning that is difficult to be achieved successfully ^{[11][12][22][23][24]}. The main approaches for giving support in IPS instruction and the outcomes obtained are the following five: driving questions, prompting, content representations tools, processing worksheets, and writing and communicating support.

1.2.2. Embedded instruction

Embedding IPS training within a meaningful context with domain-specific instruction has proved more effective than standalone courses ^[25][26][27]. Embedding instruction has the potential to increase engagement, motivation, transfer, and deep learning [49]. Previous studies investigating embedded instruction have shown good results in primary education ^[28][29], secondary education ^[12][21][30], and higher education ^[13][31].

A literature review offers theoretical and empirical evidence on the effectiveness of whole-task and embedded IPS instruction. However, there are still scarce studies combining these two key instructional approaches. For instance, ^[10] investigated an embedded IPS course designed according to a whole-task approach and instructed ten student teachers in a quasi-experimental intervention research, finding positive results in the development of IPS skills and task performance. In another study, ^[20] successfully applied an embedded IPS instruction with psychology students. In this study, students obtained good learning outcomes and increased their frequency in some of their IPS constituent skills and regulation activities. More recently, ^[32] investigated student teachers' IPS skills through an embedded whole-task instruction in a 20-week course and reported that the instruction succeeded in developing cognitive strategies to tackle an information problem.

1.2.3. Long-term instruction

Long-term instruction for learning has been considered as the instruction that lasts over a quarter of the academic year ^[33], or even as the instructional course that may take place over two or three weeks ^[34]. In the specific field of IPS, a long-term instruction has been related with a curriculum-wide approach ^[6][19][32]. Most IPS intervention studies apply short term instructions and these studies report that some of the improvements on IPS skills reached by the participants disappeared after completing the course ^[10]. In this vein, researchers claimed the need of “a scaled-up version with more content, more task classes containing tasks of increasing complexity, offered over a longer period of time and embedded in a multitude of contexts, might prove very effective.” ^[3] (p. 101). This claim is also shared by other studies, in which it is assumed that the whole-task approach to complex learning requires more learning tasks over longer periods than other kinds of instruction, but such practice will lead to better transfer to new settings when designed and conducted  adequately ^[10][35].

In summary, despite the existence of studies confirming that embedded whole-task IPS instruction improves the students’ IPS skills, there is still the need to know to what an extent the period of instruction of the IPS skills might have a positive impact on students’ learning and performance results ^[36][22][37]. Furthermore, while most educational institutions acknowledge that IPS is an essential academic skill in this digital and knowledge era, they struggle with its implementation, and specifically in finding a suitable place and sizeable time in the curriculum for IPS integration ^[3][38]. IPS skills require domain-specific knowledge and in order to guarantee their transfer to daily activities, long-term, embedded, and supported IPS practice throughout the whole curriculum is needed ^[13][39].

Notwithstanding this necessity, most IPS instruction is often implemented as a separate course and loosely connected to the curricular contents (e.g. ^[11]) and secondary education students still face difficulties in their daily school activities ^[40][41]. Therefore, it is desirable to further investigate how to embed IPS research and instruction in real secondary classrooms and learn curricular contents to provide best practices, approaches and conclusive results of quality education for all students. To this end, this paper tackles this objective and provides answers to this educational challenge by discussing the design, development and empirical testing of a long-term, embedded, whole-task IPS instruction in secondary education. Specifically, our research investigates the longitudinal effects of a three-year IPS instruction on students’ task-performance when solving complex digital problems.

1.3. The study

The present study is grounded on research by ^[32] who started to investigate the effects of long-term, embedded, whole-task instruction on the development of IPS skills in secondary education. Our study then follows up on this research and takes a longitudinal approach that aims to answer the following question: what are the effects of a long-term, embedded and whole-task IPS instruction on students’ task performance? While research shows discrete short-term learning effects, it is unclear whether there may be higher potential in a long-term situation ^[3][10]. Our study aims to contribute with new data. With this purpose, a three-year embedded IPS skills intervention was designed, during which students solved whole-task projects related to daily life challenges as well as Science, Technology, Mathematics –STEM- and Social Sciences curricular contents. In our quasi-experimental design, the digital task-performance of students following the regular curriculum (i.e. control group) was compared to that of students following the three-year IPS instruction (i.e. experimental group).

As this long-term training makes use of whole-tasks that address and support all constituent IPS skills, our expectations are that those students who follow IPS instruction will display deeper meaning-making from digital sources and better task performance than their counterparts who follow the regular curriculum. With a view to obtaining a more detailed description on the effects of IPS instruction on task performance results, the four evaluation tests carried out to assess the task performance over the three-year project included three different tasks of varying difficulty, namely: (a) fact-finding task, (b) information-gathering task, and (c) final essay. Our research aims to confirm or reject the next four hypotheses:

H1: Students following IPS instruction will produce a better fact-finding task, as measured by the number of correct answers presented within.

H2: Students following IPS instruction will solve an information-gathering task better, as measured by the number of correct argumentative concepts presented within.

H3: Students following IPS instruction will perform a better final essay, as measured by the level of explanation of the ideas written up.

H4: There will be longitudinal differences between the two groups -control and experimental groups- on IPS task performance throughout the three-year project. These differences will be more noticeable in more complex tasks that involve information-gathering as well as in the final essay.

2. Materials and Methods

2.1. Participants

The participants of our study were involved in a larger research project that aimed to promote digital literacy on secondary education students and in real-life classroom settings. For this reason, we only recruited a sample of sixty-one students (32 girls and 29 boys). It must be said though that these participants were fully committed during our ambitious three-year IPS instruction and completed the four tests of the longitudinal research throughout this period of time. From these, 42 of them corresponded to the experimental group and followed the long-term IPS instruction, while the 19 remaining were members of the control group and followed regular classes. At the beginning of the project, the students’ ages were 12/13, and by the end of the project, their ages were between 15/16. They belonged to three urban schools from the city of Lleida (Spain). In order to preserve the natural classroom environment and due to ethical issues to ensure that all students of the same school could benefit from our long-term IPS instruction, one school was established as the control group, while the other two schools acted as the intervention group.

In the control group, the students did not follow the IPS instruction and this group was used to study the natural development of IPS skills by a group of students who live in a digital society and use digital information in their daily life. Therefore, the control group students were free to use internet information to solve school assignments depending on the teacher’s learning objectives. However, the teachers did not provide guided internet use and neither did students participate in any specific instruction, course or workshop related to IPS skills nor internet navigation. 

2.2. Study design and procedure

This is a longitudinal study with a quasi-experimental design, including an experimental group, i.e. the group of interest, and a control group, used to establish the natural development of students who did not receive explicit IPS instruction. All the participants completed four tests carried out at four different moments during the three-year project.

The longitudinal study design process consisted of three main actions:

• Action 1. Initial evaluation of control and experimental students at the beginning of the research project, namely, Test 1.
• Action 2. Implementation of the IPS instruction: only the experimental group followed the three-year intervention.
• Action 3. Follow-up evaluation: at the end of every academic year, control and experimental students were tested, namely, Test 2, Test 3 and Test 4.

To carry out the three-year project, we also counted on the collaboration of eighteen secondary teachers of four school disciplines (namely, Science, Technology, Maths -STEM- and Social Sciences) who worked hand in hand with our research group in designing the IPS tasks, the supporting tools provided for each task to promote the development of IPS skills, and the embedding of the digital tasks in the school curriculum. During this collaboration, our research group ensured that the teachers became aware of the importance of IPS skills to better solve information problems and promote them in a real-life classroom setting. 

2.3. Materials and characteristics of the IPS instruction

The long-term, embedded and whole-task IPS instruction consisted in the resolution of 24 web-based learning tasks. Each task consisted in an authentic, ill-structured whole task embedded within four sessions of 60 minutes each. Therefore, students received approximately 96 hours of sustained and maintained IPS instruction for a period of three years.

All the IPS learning tasks were designed following the three key instructional and methodological principles grounded on IPS literature review presented previously: long-term, embedded in authentic curricular tasks, and whole-task instruction. Students solved problems and challenges in which they covered the entire IPS process and had to use all the constituent skills and subskills from beginning to end, including practice of IPS as a whole process in which one skill was  coordinated and integrated with the rest of the skills [10]. The instruction provided students with the 5-step structure, whereby each skill and its respective subskills functioned in an ordered and iterative way [65]. In addition, the learning tasks designed during the longitudinal project were optimised by means of technological support displayed on screen during the learning process. All these guaranteed students proper guidance and support to learn specific IPS skills and subskills [11,12,35] and the following five types of educative supports were included: driving questions, prompting, content representations tools, process worksheets, and writing and communicating support. Figure 2 shows and illustrates the five types of educational supports designed to promote the IPS development.

Figure 2.

Support provided during the information problem solving instruction.

2.4. Data collection

The instrument to collect the data of this study was a web-based authentic whole task that required unfolding learning skills. We designed two versions of the web-based task; both were about astronomy and were similar in terms of style, complexity and structure. Web-based task 1 was used as the basis for Test 1 and Test 3; its content was about planet Mars. Web-based task 2 was used as the basis for Test 2 and Test 4, its content being the Moon. All the participants solved the web-based tasks individually, in a real classroom context within 50 minutes’ time and the students’ answers were all stored in a webserver.

2.5. Measurements

In order to obtain a more detailed description about the effects of the IPS instruction in solving learning tasks of different levels of complexity, the design of the web-based task was divided into three inter-related tasks of varying difficulty, namely: (a) fact-finding task, this task focused on searching and locating relatively simple pieces of information and was usually found in one single website [61]  (b) information-gathering task, this task encouraged students to gather and integrate information from different web sources to find an answer; and (c) final essay, this task is a conclusion or sum-up task whereby each student was asked to write a short argumentative essay (of approximately 300 words) that integrated and used comprehensive web information to write a personal argument

2.6. Data analysis

The statistical methods used in the analyses were described as follows:

1. Descriptive statistics were computed for each dependent variable and each test. For quantitative variables, the following were shown: median, mean, minimum and maximum values. For qualitative variables, the following were shown: counts and percentages.
2. In order to compare the results shed by the control and intervention groups, each test was subjected to a bivariate analysis. A non-parametric Wilcoxon test for comparing medians was used for quantitative dependent variables, and a Chi-square test for qualitative dependent variables.
3. To analyse the longitudinal effect of long-term IPS instruction on the different dependent variables, a general linear model with repeated measures, e.g., [68], was established for the quantitative dependent variables. The results were given in terms of least squares means differences. For qualitative nominal dependent variable, on account of three nominal categories (no answer – facts – explanation), three logistic regression models, e.g., [69], were established. For these models the results were displayed using the odds ratio (OR) and the corresponding 95% confidence interval reported.

The statistical significance was defined as p<0.05. All the results were obtained using SAS software, v9.3. Copyright, SAS Institute Inc., 2007.

3. Results

We have organised the presentation of the results in two subsections: (1) descriptive results obtained by the control and experimental group students in IPS task-performance variables in each test and bivariate analyses; (2) longitudinal analyses of long-term instruction effects on task performance.

3.1. Descriptive results and bivariate analyses

The results presented in Figure 3 show that both groups of students obtained lower scores in information-gathering than in fact-finding tasks (in line with hypotheses 1 and 2). From these results we can infer that information-gathering tasks are more complex and difficult than fact-finding tasks.

Figure 3. Means of fact-finding and information-gathering task-performance results (dependent variables 1 and 2); * stands for: p<0.05.

As the project progressed (see Figure 4), both groups, on average obtained better scores in fact-finding as well as in information-gathering tasks. However, this increase was significantly higher in the experimental group that followed the long-term IPS instruction. The experimental group also performed better in solving the information-gathering tasks; thus, from Test 1 through to Test 4, the median increased by 3.3 points (out of 7) in this type of task. By contrast, the increase for the control group students in this task was only of 2 points.   

 Figure 4 shows the results of the different categories identified in the final essay, these results show that at the outset of the study, any student had Explanation category in Test 1. As the project progressed, we identified different pattern responses in control and experimental students. While the experimental group students reduced drastically the percentage of No answer category and increased firmly the percentage of Explanation category (from 0 in Test 1 to 40.48% in Test 4), the control group students maintained high rates of No answer and very low rates of Explanation throughout the project.

Figure 4. Final essay task-performance (dependent variable 3): percentages of qualitative nominal categories.

Bivariate analyses were carried out to study the effect of IPS instruction on students’ IPS task-performance in each evaluation point and the results point out that those students who followed the long-term, embedded, whole-task IPS instruction showed higher improvement on the performance of complex IPS tasks and were more capable to generate explanations from web information than control group students.

3.2. Longitudinal analyses of IPS instruction effects on task performance

A stratified general linear model with repeated measures analysis was used in order to investigate how the long-term, embedded, whole-task IPS instruction improved the experimental students’ IPS task-performance throughout the project. The longitudinal analysis points out that IPS instruction has a high statistically significant impact on the evolution of experimental students’ IPS fact-finding and information-gathering task-performance (hypotheses 1 and 2).

Control group students also showed a longitudinal increment in IPS task performance scores; however, these increments are lower than the ones obtained by intervention group students.

To verify whether there were any statistical differences between control and intervention group on IPS task performance scores throughout time (hypothesis 4), a Stratified General Linear model with repeated measures was performed. Statistically significant differences between the two groups were found in information-gathering task-performance; the estimated LSM difference between the two groups in this variable was 0.9695 [-0.9695, CI 95% = (-0.386, -1.553)]. However, no statistically significant differences between control and experimental groups were found in fact-finding task performance. This result reveals that the positive longitudinal effect of long-term IPS instruction was higher when solving complex IPS tasks.

A logistic regression model was carried out to investigate the longitudinal effect of long-term, embedded, whole-task IPS instruction on final essay task-performance (hypothesis 3). As this is a qualitative nominal variable with three nominal categories (No answer, Facts, Explanation), three logistic regression models were established. Model 1 analysis (No answer vs Answer) showed that in the control group the odds ratio of No answer was almost two times higher than that of the intervention group. Model 2 analysis (No answer versus Explanation) showed that the odds of obtaining No answer in the control group were five times higher than in the intervention group. Finally, Model 3 analysis (Facts vs Explanation) revealed that intervention group students were 3.39 times more likely to generate explanation responses in their essays than control group students. This finding shows a positive longitudinal effect of IPS instruction on integrating and using web information comprehensively.

4. Discussion and conclusions

Even though schools experience difficulties in embedding IPS skills across the curriculum, our study successfully implemented and evaluated a long-term, embedded, whole-task IPS instruction in real-life classroom settings. Consequently, the instructional approach used in this study can make a valuable contribution in providing good practice and quality education for all students and to overcome some of the difficulties stated by previous research ^[41][42].

Moreover, this paper has researched the longitudinal effect of IPS instruction on the students’ task-performance and learning in real-life classroom settings. Our results reveal that long-term, embedded, whole-task IPS instruction has a high positive effect on the students’ IPS performance as time goes by. Thus, students following IPS instruction produced a better fact-finding task (hypothesis 1), information-gathering task (hypothesis 2), and a better final essay (hypothesis 3), as verified by the quality of the explanation of the ideas expressed. In addition, we found longitudinal differences between control and experimental group as regards IPS task-performance throughout the three-year project, because instructed students outperformed their counterparts in more complex tasks, namely information-gathering task and final essay (hypothesis 4).

These findings coincide with the results obtained in previous research in which the participants’ engagement in web-search instruction could improve their learning and efficiency in solving the task ^[12][30]. Other authors also pointed out that the better the strategy and reflection on each skill, the better the efficiency, e.g., ^[36][18].

Our results also reveal that, as the project progressed, those students who followed the long-term, embedded, whole-task IPS instruction solved information problems better than the control students. Hence, the positive effects of the students’ participation in IPS instruction are higher in solving complex tasks. In information-gathering tasks, whereby complex cognitive processes come into play and encourage students to extract meaning out of complex digital documents, experimental students outperformed control students as the project progressed and these differences increased over time. This is in line with previous studies that had shown that the more complex the task, the deeper the processing of information. Likewise, a more expert pattern was required to solve the problem successfully ^[10][22][23]. We can then conclude that the IPS instruction designed in this study promoted the students’ development of those informational skills that helped them learn from internet resources.

Besides, our research contributes with experimental evidence to the fact that the IPS instruction designed in this study helped students to generate more and better explanations in their final essay. Thus, our students learned to integrate information from digital resources and build meaning-making to provide a successful answer to an informational problem. Previous research had already demonstrated that the process of explanation increased critical thinking and understanding by pushing an agent to explain the consequences of his/her view, to search new information needed for answering questions and achieving his/her cognitive goals [70]. To conclude, our study extends on previous results and gives experimental evidence that long-term, embedded, whole-task instruction in real-life classroom settings is desirable to efficiently help students to learn while solving complex digital tasks.

In the light of the results obtained in our study we can claim that the embedded support across long-term instruction boosts the students’ IPS skill development. The students receiving the instruction mastered IPS skills that helped them draw relevant conclusions out of internet resources and succeeded in using this information meaningfully to construct their own arguments in the final essay. This confirms previous studies by ^[43] in that such factors as the students’ use of adequate search strategies, the adoption of assessment strategies towards online information, and the quality of online resources obtained by students were essential to explain the successful development of science-related conceptual understandings expressed in a final essay. Along the same line of argument, ^[11] related greater source evaluation reached by instructed students in a secondary school with a deeper level of information comprehension to solve an information task.

Our study strengthens the claim highlighted by previous research in which natural interaction with digital information and non-explicit support is not enough to furnish our students with those IPS skills needed to solve complex information problems ^{[3][14][19][44]}.