January 2020: Are Reading Interventions for ELLs Effective?

Study Background

Ludwig and colleagues (2019) recently conducted a meta-analysis of reading interventions for English language learners (ELLs). ELLs are students who first learn to speak, read, and/or write a language other than English. A meta-analysis is a systematic review that involves a statistical technique for combining the results of several empirical studies into a single quantitative estimate of an effect (Petticrew & Roberts, 2008).

Recent statistics and evidence have shown that ELLs have weaker literacy skills and are often at risk of falling behind academically (Kieffer, 2011; Lesaux et al., 2007; National Center for Education Statistics, 2015). This highlights the necessity to examine the effectiveness of reading interventions for ELLs. Although reading interventions for the general student population have been systematically examined extensively, only one meta-analysis has been conducted to examine the reading interventions for ELLs. (Richards-Tutor et al., 2016). However, the meta-analysis conducted by Richards-Tutor and colleagues (2016) included only experimental studies with ELLs who were either at risk for reading difficulties or had a diagnosis of a learning disability. Results showed small effect sizes for phonics, fluency, and comprehension measures. They also examined moderating variables: factors that could influence the effectiveness of intervention, such as group size or time of instruction, and found no significant effects of various moderating variables. Previous meta-analyses with the general student population have revealed that certain variables do moderate the effect of intervention, with such factors including the intensity of intervention, group size, type of intervention, and the students’ risk status (Ehri et al., 2001; Suggate, 2016; Wanzek & Vaughn, 2007). In summary, the effect of reading interventions for ELLs need to be further explored through experimental studies with ELLs, regardless of at-risk status.   

Purpose and Research Questions

The purpose of the meta-analysis conducted by Ludwig et al. (2019) was to examine the impact of reading interventions on ELLs’ reading skills and the factors that may influence their effectiveness. Two research questions were specifically addressed:

1. What is the size of the effect of reading interventions on ELLs’ reading skills?
2. Does group size, intensity of intervention, students’ risk status, or type of intervention account for some of the variability in the effects of the interventions?

Methodology

Studies were included if they met the following criteria: (a) included participants identified as ELLs and assessed on reading accuracy, reading fluency, and/or reading comprehension; (b) involved an intervention targeting a reading-related skill over a period of time (more than a single session) at school during regular school hours in addition to the general education curriculum; (c) employed an experimental or quasi-experimental design and reported pre- and posttest means, standard deviations, and sample sizes; and (d) involved participants enrolled in kindergarten through Grade 12 in a school program in which the primary language of instruction was English.

Twenty-six studies met inclusion criteria. To estimate the effect size of the interventions, the standardized mean change for each reading outcome was calculated. Six moderator variables were coded to examine factors that may influence the effect of interventions. These variables included number of sessions per week, the length of each intervention session, total length of the intervention, group size, students’ risk status, and type of intervention. Students’ risk status included at risk or not at risk for reading difficulties. Type of intervention included phonological awareness, phonics, vocabulary, and multicomponent interventions.

Overall, the majority of studies (53.8%) examined the effects of interventions provided in either kindergarten or Grade 1. The interventions were provided to typical readers in 12 studies, to at-risk readers in 12 studies, and to both typical and at-risk readers in 2 studies. The frequency of the intervention was reported in 24 studies and ranged from two to five sessions per week. The duration of each session ranged from 12.5 to 90 minutes, with sessions lasting 30 minutes or less occurring most frequently (69.2%). Dependent variables (accuracy, fluency, comprehension) varied across studies, with four studies reporting results for all three variables, five reporting for only two variables, and 17 studies reporting on only one variable.

Key Findings

1. What is the effect size of reading interventions on ELLs’ reading skills?

• The results revealed large effects for reading accuracy (d =1.22) and fluency (d = 0.802) and moderate effects for reading comprehension (d = 0.499). All estimated effect sizes were larger than those reported in Richard-Tutor et al.’s (2016) meta-analysis, as well as larger than those reported in previous meta-analyses with studies from the general student population (Ehri et al., 2001; Ehri et al., 2001; Scammacca et al., 2015; Suggate, 2016; Wanzek et al., 2016). (An effect size is an estimate of the impact of an intervention or treatment. Further information for interpreting effect sizes can be found in the interactive resource “Interpreting Cohen’s d Effect Size” [Magnusson, n.d]. A confidence interval describes a range of values that will on average contain the true value with a certain degree of probability. Further information about confidence intervals can be found in “Quantifying Effects and Designing Studies” [Poldrack, 2020]).
• Overall, the effect of reading accuracy and reading fluency interventions were stronger and more promising than reading comprehension interventions. To confirm the effectiveness of reading interventions for ELLs, further studies are needed.

2. Does group size, intensity of intervention, students’ risk status, students’ grade level, or type of intervention account for some of the variability in the effects of the interventions?

• Overall, none of the moderators were statistically significant when explaining variability between studies. Looking at studies that included measures of real-word reading, two significant moderators, the group size and the length of each intervention session (in minutes), were found.
  • Intervention groups composed of more than five students were less effective than groups composed of two to five students.
  • Longer intervention sessions were less effective than shorter ones. However, this effect was likely driven by two studies in which longer interventions were provided to large groups (Lovett et al., 2008; Rodríguez et al., 2012).
• Contrary to previous meta-analyses (Scammacca et al., 2015; Suggate, 2016), students’ risk status was not a significant moderator. This result might be due to the fact that the criteria for coding “at risk” or “not at risk” varied across studies. The authors also found that averaged effect sizes were larger for kindergarten and Grade 1 than for older grades, but this result was not statistically significant. Furthermore, the authors suggested that interventions with multiple components are not necessarily superior to those focusing on only one component (i.e., phonological awareness), because of the nonsignificant moderating effect of intervention type.
• However, it is not possible to make meaningful comparisons due to the small number of studies, as the effect of intervention may be underestimated.
• To clarify which type of intervention is better for ELLs, further studies are needed.

Limitations

A major limitation of this meta-analysis was the heterogeneity across samples, including instructional differences, individual differences (e.g., coding of students’ risk status), and reading task differences. With regard to instructional differences, the authors acknowledged that the specific reading interventions contained within included studies varied widely and were not discussed in detail, which hinders the attempt to define which specific instructional practices in accuracy, fluency, or comprehension were most effective for ELLs across grade levels. Another example of heterogeneity that affects interpretation of the meta-analysis was variation across studies of the type of reading outcome measure. This can affect the results, because the type of outcome measure, whether it is standardized or researcher-developed, can influence effect sizes. A final limitation was that some moderators could not be fully investigated due to the small number of studies included in this study.

Implications for Practitioners

For practitioners and policy makers, the findings of this meta-analysis suggested that ELLs can benefit greatly from a wide variety of reading interventions, particularly in reading accuracy and reading fluency. The authors suggest that the large effect sizes obtained indicate that the key components of literacy instruction (e.g., phonological awareness, phonics, fluency, vocabulary) work equally well with ELLs. The authors further recommend not delaying intervention to ELLs until these students reach a certain level of oral English proficiency. Given that short intervention sessions with small groups were more effective than long intervention sessions with large groups for studies of word reading, it might be efficient for teachers to deliver the intervention with a group composed of two to five ELL students in short intervention sessions.

This meta-analysis provided some evidence of intervention effect for ELLs. Nevertheless, due to certain limitations of the meta-analysis, further research and evidence are needed.

References

Ehri, L. C., Nunes, S. R., Stahl, S. A., & Willows, D. M. (2001). Systematic phonics instruction helps students learn to read: Evidence from the National Reading Panel’s meta-analysis. Review of Educational Research, 71, 393–447.

Ehri, L. C., Nunes, S. R., Willows, D. M., Schuster, B. V., Yaghoub-Zadeh, Z., & Shanahan, T. (2001). Phonemic awareness instruction helps children learn to read: Evidence from the National Reading Panel’s meta-analysis. Reading Research Quarterly, 36, 250–287.

Kieffer, M. J. (2011). Converging trajectories: Reading growth in language minority learners and their classmates, kindergarten to Grade 8. American Educational Research Journal, 48, 1187–1225.

Lesaux, N. K., Rupp, A. A., & Siegel, L. S. (2007). Growth in reading skills of children from diverse linguistic backgrounds: Findings from a 5-year longitudinal study. Journal of Educational Psychology, 99, 821–834.

Lovett, M. W., De Palma, M., Frijters, J., Steinbach, K., Temple, M., Benson, N., & Lacerenza, L. (2008). Interventions for reading difficulties: A comparison of response to intervention by ELL and EFL struggling readers. Journal of Learning Disabilities, 41, 333–352.

Magnusson, K. (n.d.). Interpreting Cohen’s d Effect Size. Retrieved February 2, 2020, from https://rpsychologist.com/d3/cohend/

National Center for Education Statistics. (2015). Fourth-grade reading scores increase for two demographic groups compared to 2013. Retrieved February 21, 2017, from https://www.nationsreportcard.gov/reading_math_2015/#reading/groups?grade=4

National Reading Panel. (2000). Report of the National Reading Panel: Teaching children to read. National Institute of Child Health and Human Development.

Petticrew, M., & Roberts, H. (2008). Systematic reviews in the social sciences: A practical guide. Blackwell.

Poldrack, R. A. (2020). Statistical thinking for the 21st century. Retrieved February 12, 2020, from https://statsthinking21.org/sampling.html#confidence-intervals

Richards-Tutor, C., Baker, D. L., Gersten, R., Baker, S. K., & Smith, J. M. (2016). The effectiveness of reading interventions for English learners: A research synthesis. Exceptional Children, 82, 144–149.

Rodríguez, C. D., Filler, J., & Higgins, K. (2012). Using primary language support via computer to improve reading comprehension skills of first-grade English language learners. Computers in the Schools, 29, 253–267.

Scammacca, N. K., Roberts, G., Vaughn, S., & Stuebing, K. K. (2015). A meta-analysis of interventions for struggling readers in Grades 4–12: 1980–2011. Journal of Learning Disabilities, 48, 369–390.

Suggate, S. P. (2016). A meta-analysis of the long-term effects of phonemic awareness, phonics, fluency, and reading comprehension interventions. Journal of Learning Disabilities, 49, 77–96.

Wanzek, J., & Vaughn, S. (2007). Research-based implications from extensive early reading interventions. School Psychology Review, 36, 541–561.

Wanzek, J., Vaughn, S., Scammacca, N., Gatlin, B., Walker, M. A., & Capin, B. (2016). Meta-analyses of the effects of Tier 2 type reading interventions in Grades K–3. Educational Psychology Review, 28, 551–576.