HAPPY BOOK

While technology has altered how we communicate, access information, work, and even play, disrupting most economic sectors, its impact on education has been much more limited. Using technology to replace analogue tools without considering the technology's comparative advantages is primarily to blame for this little impact. Compared to conventional "chalk-and-talk" classroom instruction, these advantages include assisting in expanding standardized education, making it easier to provide differentiated instruction, expanding opportunities for practice, and increasing student engagement. Students will thrive in schools that use technology to enhance teachers' work and improve the quantity and quality of educational materials.

In addition, COVID-19 has demonstrated that, in the current climate of pandemics and climate change, schools cannot consistently provide in-person education, supporting the need to invest in education technology.

Here, we argue for a straightforward yet unusual approach to education technology that aims to:

Comprehend a school system's requirements, infrastructure, and capacity—the diagnosis;

Examine the most recent and reliable research technology for teaching and learning on treatments for those conditions—the evidence; and closely monitor the prognosis—the outcomes of innovations before they are scaled up.

Related Content: Podcast: September 11, 2020, How training innovation can further develop learning for all understudies

September 10, 2020

To make ed tech work, put forth clear objectives, audit the proof, and pilot before you scale

The system

Our methodology expands on a basic, yet natural hypothetical system made twenty years prior by two of the most conspicuous training specialists in the US, David K. Cohen and Deborah Loewenberg Ball. They argue that the interactions between educators and students regarding educational materials are the most critical factor in enhancing learning. The need for more clarity in improving the interactions between educators, students, and the educational material is similar to the ed-tech reforms implemented in most of the developing world up to this point. These reforms are identical to the failed school improvement efforts in the United States that inspired Cohen and Ball's framework. We add parents as critical mediators of the relationships between students, educators, and the material, expanding on their framework (see Figure 1).

Image 1: 

Based on Cohen and Ball's (1999) illustration of the instructional core, ed-tech interventions can have a wide range of effects on the instructional body. However, technology should only do something if it can. In developing nations, school systems differ in many ways. Each system likely has different requirements for ed-tech interventions and other infrastructure and capacity to implement such interventions.

The finding:

How can school systems evaluate their readiness and needs?

Diagnosing a school: is a helpful first step in deciding whether or not to invest in education technology.

Specific requirements for enhancing student learning, such as raising the average level of achievement, closing achievement gaps, and requiring high performers to acquire higher-order skills;

Infrastructure to implement technology-enabled solutions (such as electricity connection, available space and outlets, computer stock, and Internet connectivity at schools and in the homes of students); and the ability to incorporate technology into the teaching process (e.g., how well-versed students and teachers are with hardware and software, what they think makes technology use for learning, and how they currently use it).

School systems should fully use the administrative data already in place before starting any new data collection project that might help answer these three main questions. This could take the form of internal evaluations as well as international learner assessments like the PISA, Trends in International Mathematics and Science Study (TIMSS), Progress in International Literacy Study (PIRLS), and Teaching and Learning International Study (TALIS). We developed a set of surveys for students, educators, and school leaders; however, if school systems want to supplement existing data with a more comprehensive set of indicators or need more information on their readiness for ed-tech reforms. Download the full report to see how we outline the main points covered by these surveys to show how they can guide decisions about ed-tech interventions.

The information:

How can educational institutions find promising ed-tech solutions?

Because school systems differ regarding learners and teachers and the availability and quality of technologies and materials, no "ed-tech" initiative will work in all schools. Instead, decision-makers should concentrate on four potential uses of education technology that capitalize on educational media its comparative advantages and complement educators' efforts to accelerate student learning to realize education technology's potential (Figure 2). These immediate benefits include:

It improves quality instruction on a larger scale, such as through prerecorded quality lessons.

We are facilitating differentiated instruction, for instance, through live one-on-one tutoring and computer-adaptive learning.

We are extending practice opportunities.

We are utilizing games and videos to increase student engagement.

Image 2: 

Technology's advantages in comparison Figure 2 Technology's advantages in comparison Adapted from Cohen and Ball (1999) In this section, we organize the research on ed-tech interventions from 37 studies in 20 countries* by comparative advantage. Our classification of these interventions is one of many; for example, video tutorials could be used to scale up instruction or increase learner engagement. First, however, highlight the requirements these interventions could fulfil and why technology is well positioned to do so.

We use standard deviations (SDs) to describe the magnitude of the effects of interventions when discussing specific studies. In research, SDs are a standard metric for expressing the impact of a program or policy on a business-as-usual condition (such as test scores). They can be understood in a variety of ways. One option is to classify the effects' extent according to the impact evaluations' findings. For example, results below 0.1 SDs are considered minor in developing nations, while those between 0.1 and 0.2 SDs are considered medium. Those above 0.2 SDs are considered significant (for reviews that estimate the average effect of groups of interventions, referred to as "meta-analyses," see, for instance, Conn, 2017; 2013 by Kremer, Brannen, and Glennerster; 2014, McEwan; Snilstveit and others, 2015; 2020, Evans and Yuan.)

*In studying the proof, we started by arranging studies from earlier broad and ed-tech explicit proof surveys that a few of us have composed and from ed-tech surveys led by others. The studies cited by the ones we had previously read were then followed and reviewed. When selecting studies for inclusion, we focused on experimental and quasi-experimental evaluations of education technology interventions from preschool through secondary school in low- and middle-income countries released between 2000 and 2020. In addition, we only included interventions that aimed to improve student learning directly (students' interaction with the material), not those that indirectly impacted achievement by decreasing teacher absence or increasing parental involvement. This procedure resulted in 37 studies from 20 nations (the complete list of studies can be found in Appendix B).

Expanding standardized instruction Technology's capacity to deliver standardized, high-quality content on a large scale is one way it may improve education quality. This technology feature may be handy in three situations: a) those in "difficult to-staff" schools (i.e., schools that battle to enrol teachers with the imperative preparation and experience — commonly, in provincial or potentially distant regions) (see, e.g., Urquiola and Vegas, 2005); ( b) those in which many teachers are absent from the classroom frequently (e.g., Chaudhury, Hammer, Kremer, Muralidharan, & Rogers, 2006; Muralidharan, Das, Holla, and Mohpal, 2017); and (c) those where teachers lack pedagogical and subject matter expertise (Bietenbeck, Piopiunik, & Wiederhold, 2018; Strong et al., 2017; 2012, Metzler and Woessmann; Santibañez, 2006) and do not have chances to notice and get criticism (e.g., Bruns, Costa, and Cunha, 2018; Taylor, Cilliers, Fleisch, and Prinsloo, 2018). The technology could solve this issue in the following ways: a) distributing lessons to many students through prerecorded or live lessons taught by qualified educators, b) allowing students in remote areas and during school closures to participate in distance education, and (c) distributing educational-content-loaded hardware.

Prerecorded lessons Technology is in an excellent position to spread effective educators' lessons and increase their impact. Although inconclusive, the evidence regarding the impact of prerecorded lessons is encouraging. For example, student learning on independent assessments has increased as a result of some initiatives that have utilized short instructional videos in addition to other learning materials to supplement regular instruction. Beg et al., for example, 2020) looked at an initiative implemented in grade 8 classrooms in Punjab, Pakistan. The intervention consisted of short videos that were used in place of live instruction, quizzes that allowed students to practice the material from each lesson, tablets that allowed teachers to learn the material and follow the lesson, and LED screens that projected the videos onto the screen in the classroom. After six months, learners' scores on independent math and science tests increased by 0.19 and 0.24 standard deviations, respectively. However, the intervention did not affect Punjab's high-stakes math and science sections.

One study suggests that less technologically advanced approaches can improve learning outcomes, mainly if business-as-usual instruction is better. For instance, Naslund-Hadley, Parker, and Hernandez-Agramonte (2014) looked at a preschool math program in Cordillera, Paraguay, that used written and audio materials four days a week for an hour each day during the school day. As a result, math scores improved by 0.16 standard deviations after five months, closing the achievement gap between low- and high-achieving students and educators with and without formal early childhood education training.

However, not all attempts to incorporate prerecorded content into regular instruction have been successful. For instance, de Barros (2020) looked at an intervention in Haryana, India, that included instructional videos for math and science, upgrades to the infrastructure (such as two "smart" classrooms, two TVs, and two tablets), printed workbooks for students, and in-service training for educators of students in grades 9 and 10 (all of the materials were mapped to the official curriculum). The intervention did not affect science (in business-as-usual classes) or math achievement after 11 months (by 0.08 standard deviations). However, it harmed an index of instructional quality and decreased the proportion of lesson time teachers devoted to instruction. Similarly, Seo (2017) looked at several different combinations of infrastructure (such as TVs and solar lights) and prerecorded English or bilingual videos for 11th-grade students in northern Tanzania. They found none of the combinations improved learning, even when the videos were used. As others have pointed out, this method of estimating impact is problematic (Muralidharan, Romero, & Wüthrich, 2019), as the study reports effects from the infrastructure component across variants.

However, a very similar after-school intervention significantly affected students' fundamental skills. For example, Chiplunkar, Dhar, and Nagesh (2020) assessed a drive in Chennai (the capital city of the territory of Tamil Nadu, India) conveyed by the very association that joined brief recordings that made sense of key ideas in math and science with worksheets, facilitator-drove guidance, little gatherings for shared learning, and periodic vocation directing and direction for grade 9 understudies. Five times per week, these lessons took place after school for one hour. After ten months, it significantly impacted student achievement on basic math and reading skills tests. However, it did not affect socio-emotional skills or a standardized high-stakes test in 10th grade.

There are at least two reasons why drawing general conclusions from this body of research is challenging. First, the impact of several other components (such as hardware, print materials, or other activities) in conjunction with prerecorded lessons has been examined in the studies mentioned above. Therefore, it is possible that these additional components, rather than the recordings themselves or their interaction with one another, are to blame for the observed effects (for discussing the difficulties associated with interpreting "bundled" interventions, see Muralidharan, 2017). Second, these studies evaluate prerecorded lessons somehow, but none look at their content. As a result, it seems entirely plausible that the direction and magnitude of the effects are primarily determined by the quality of the recordings (such as the educator's expertise, the amount of planning that went into the recording, and its alignment with best teaching practices).

Additionally, these studies bring up three significant concerns that should be investigated in subsequent research. One of them is why, even though their materials are typically mapped onto the official curriculum, none of those mentioned above interventions affected high-stakes exams. According to Pritchett & Beatty (2015), the official curricula may be too difficult for students in these settings, who frequently require primary skill reinforcement and are several grade levels behind expectations. The question of whether these interventions have a lasting impact on teaching methods is another. Assuming these mediations are sent in settings with low showing quality, teachers might gain something from watching the recordings or paying attention to the accounts with students. One more inquiry is whether these intercessions make it simpler for schools to convey guidance to students whose local language is other than the authority vehicle of guidance.

Distance education can also be provided by technology to students in rural areas. These initiatives evidence is encouraging. For instance, Johnston and Ksoll (2017) evaluated a program that offered rural primary school students in Ghana's Volta and Greater Accra regions live instruction via satellite. In order to accomplish this, the program also provided classrooms with solar panels, a satellite modem, a projector, a webcam, microphones, and a computer with interactive software to connect to a studio in Accra. According to school visits, the intervention had no effect on attendance or classroom time devoted to the instruction after two years. However, it did raise the numeracy scores of students in grades 2 through 4 and some foundational literacy tasks. The authors interpreted these findings as implying that, rather than an increase in instructional university of management and technology time, the improvement in children's achievement may have been the cause of the gains. A similar program in the Indian state of Karnataka was evaluated by Naik, Chitre, Bhalla, and Rajan (2019) and found to affect learning outcomes positively. However, whether those effects are due to the program itself or differences in the groups of students, they compared to estimate the initiative's impact is still being determined.

This kind of distance education had positive long-term effects in one setting (Mexico). In order to estimate the impact, Navarro-Sola (2019) made use of the staggered rollout of the telesecundarias, which were middle schools with lessons broadcast via satellite TV, in 1968. The policy caused short-term effects on student enrollment: Ten students enrolled in middle school, and two pursued further education for every 50 children infected with telesecundaria. Additionally, it had a long-term impact on the graduates' educational and employment paths. The policy increased average income by nearly 18% for each additional year of education. The shift from agriculture and the informal sector to more graduates entering the workforce was the cause of this effect. In a similar vein, Fabregas (2019) utilized a subsequent expansion of this policy in 1993 to discover that each additional telesecundaria per 1,000 adolescents resulted in an average increase of 0.2 years of education, a decrease in women's fertility, and no conclusive evidence of long-term effects on outcomes in the labour market.

It is essential to interpret these findings considering the contexts in which the interventions were carried out. As was mentioned earlier, one reason they have been successful is that the "counterfactual" conditions for learning—that is, what would have happened to students without such programs—were either lack of access to education or low-quality instruction. It is essential for school districts interested in implementing similar interventions to determine to what extent their students, or portions of their student body, are experiencing conditions comparable to those of the subjects of the previous studies. This exemplifies the significance of determining a system's requirements before examining the evidence.

Preloaded equipment

Innovation is strategically situated to scatter informational materials. More specifically, educational software, such as word processing, reference books, and games, may also be delivered utilizing hardware like desktop computers, laptops, or tablets. Theoretically, these materials could not only undergo a quality assurance review (such as by curriculum specialists and educators), but they could also draw on student interactions for adjustments (such as determining areas that require reinforcement) and facilitate interactions between students and educators.

However, in practice, most initiatives that have given students free computers, laptops, and netbooks must take advantage of the abovementioned opportunities. Instead, they install a standard set of educational materials and hope students will use them independently. According to Malamud & Pop-Eleches (2011), students rarely do so and instead use laptops for leisure activities, frequently to the detriment of learning. Free netbook programs have not only consistently failed to raise students' math and language arts achievement (e.g., Cristia et al., 2017), but they have not affected students' general computer skills (Beuermann et al., for example). 2015). The mechanisms by which some of these initiatives have affected cognitive skills in a small way are still unknown.

According to our knowledge, the only instance in which a free laptop program was implemented successfully was when a group of researchers installed remedial software on the computers. For example, Mo et al. ( 2013) evaluated a version of the One Laptop Per Child (OLPC) program for third-grade students attending migrant schools in Beijing, China. The laptops were loaded with remedial math software mapped to the national curriculum (similar to the software products we discuss under the heading "practice exercises" below). After nine months, the program increased computer skills by 0.33 SDs and math achievement by 0.17 SDs. This study suggests that the quality of the laptop software is crucial if a school district decides to invest in free laptops.

However, the evidence does not indicate that children learn more from interacting with laptops than from textbooks. For instance, Bando, Gallego, Gertler, and Romero (2016) compared the effects of providing free laptops and textbooks to 271 elementary schools in disadvantaged Honduran regions. Students in grades 3 and 6 who received laptops performed similarly to those who received textbooks in math and language after seven months. In addition, the costs of providing laptops—not just the hardware, but also the associated technical assistance, Internet, and training—are not yet low enough to make them a more cost-effective method of delivering content to students. This is the case even though textbooks essentially become obsolete at the conclusion of each school year, whereas laptops can be reloaded with new materials for each year.

There is limited but encouraging evidence that tablets with the software are available. Take, for instance, de Hoop et al. 2020) looked at a composite intervention for first-graders in Zambia's Eastern Province that combined educational materials, hardware (projectors and tablets), infrastructure (electricity generated by solar power), and infrastructure (lesson plans for teachers and interactive lessons for students, both loaded onto tablets and mapped onto the official Zambian curriculum). The intervention increased the students' early-grade reading, oral vocabulary, and early-grade math scores by 0.22 standard deviations over 14 months. Additionally, it resulted in a 0.16 increase in students' achievement on a locally developed test. However, it is difficult to identify the components driving the positive effects due to the program's multifaceted nature. Pitchford (2015) looked at an intervention that gave students in Lilongwe, Malawi, tablets with educational "apps" to use for 30 minutes a day for two months to improve their early math skills. Positive effects on math achievement were found in the evaluation, but the primary limitation of the study is that it was only conducted in one school.