Wanzek, J., Petscher, Y., Otaiba, S. A., Rivas, B. K., Jones, F. G., Kent, S. C., . . . Mehta, P. (2017). Effects of a year long supplemental reading intervention for students with reading difficulties in fourth grade. Journal of Educational Psychology. Advance online publication. doi:10.1037/edu0000184
Summary by Dr. Jamie M. Quinn
Background
Research on the effectiveness of interventions for students with reading difficulties in the upper-elementary grades is limited compared to research with students in the early-elementary years (Wanzek, Wexler, Vaughn, & Ciullo, 2010). This limited research, as summarized by Wanzek et al. (2010), indicates that interventions for upper-elementary students have large effects for reading comprehension instruction, but the authors note that most of these interventions examined effects on proximal measures created by the research team that conducted the study. This confound is notable, as Scammacca et al. (2007) found that researcher-developed measures produce larger effect sizes than standardized measures of the same construct.
Most of the upper-elementary interventions were brief and limited in scope to only one instructional strategy. Yet multicomponent interventions, when implemented properly and with fidelity, often show larger effects than single-component programs. The few multicomponent studies in this analysis included reading comprehension and fluency and/or vocabulary instruction, with average effect sizes ranging from 0.37 to 1.87 (Wanzek et al., 2010).
Further, there is limited research on the effectiveness of interventions for students with varying levels of English language proficiency. English learners may be at a higher risk for developing late-emerging reading difficulties (after third grade), which may reflect struggles with syntax, vocabulary, and background knowledge required to comprehend increasingly complex texts.
Passport to Literacy
Passport to Literacy is a multicomponent intervention for reading that is widely used in the United States. It includes semi-scripted lessons of explicit instruction and strategies for decoding, word reading, comprehension, and vocabulary. These lessons help the reader engage with complex text by improving foundational reading skills, building background knowledge, and incorporating strategies for comprehension. A pilot study with 221 students indicated no effects on word reading or fluency but small effects on standardized reading comprehension measures (d = 0.14–0.28). However, the findings were moderated by comprehension skills the intervention was least effective for students with low levels of reading comprehension.
Study Purpose and Methodology
The purpose of Wanzek et al.’s (2017) study was to implement Passport to Literacy with students with comprehension difficulties to examine its effects on word reading, vocabulary, and reading comprehension skills. Additionally, the authors used initial reading achievement levels and English language proficiency levels to investigate differential effects of the intervention.
The authors selected fourth-grade students (n = 451) who scored below the 30th percentile on the reading comprehension portion of the Gates-MacGinitie Reading Test from 16 public elementary schools in six school districts. The sample was 49% male, 35% black, 44% white, and 17% American Indian; 46% of the students identified as Hispanic. More than 13% of the students used a language other than English at home or received English language services. Approximately 15% had a learning disability or speech/language disability. The study had low attrition, according to the What Works Clearinghouse (2014).
Students were randomized within school to the treatment with Passport (n = 226) or to a comparison group (n = 225). The researchers collected measures of word reading, reading comprehension, reading fluency and accuracy, and vocabulary knowledge at pretest and at posttest. The researchers also collected data regarding the classroom instruction practices (Tier 1) and evaluated fidelity to the Passport to Literacy intervention via direct monthly observation. The Passport to Literacy intervention was implemented in small groups outside of core classroom reading instruction for approximately 30 minutes. Although both groups received nearly equivalent instruction in phonics and word recognition, spelling, and reading fluency, students in the Passport condition received more instruction in vocabulary knowledge/oral language and reading comprehension. Alternatively, school-provided intervention covered more noninstructional text reading and other academic instruction.
The authors used a multilevel, longitudinal structural equation model (ML-SEM) to estimate main and conditional impacts of the intervention. The n-level SEM (Mehta & Neale, 2005) analysis considered the partially nested structure of the data, such that students in the experimental condition were nested in small groups, but students in the comparison condition were not nested.
Study Findings
- Students in the Passport group had significantly higher reading comprehension (α = 1.26) than those in the comparison group (α = 0.88) for an effect size difference of 0.38. There were no significant differences between groups on the measures of word reading (Δα = 0.06) or vocabulary knowledge (Δα = 0.08).
- Letter-word identification significantly moderated reading comprehension outcomes, such that students with higher initial word reading scores performed better on reading comprehension after intervention.
- There were no significant moderated effects for either word reading or vocabulary outcomes.
- English learners benefited from Passport to Literacy with regard to reading comprehension (Δα = 0.19) and word reading (Δα = 0.38) but not vocabulary (Δα = 0.01).
Interpretation of Study Findings and Implications for Future Research
Wanzek et al. (2017) extended prior work on interventions for upper-elementary students by using a large sample of students with reading difficulties and using latent variables in the estimation of treatment effects. The intervention increased the reading comprehension skills of students with comprehension difficulties and was effective for students with limited English proficiency. However, students in the Passport for Literacy sample did not increase word reading or vocabulary knowledge more than students in a comparison group, with the exception of word reading for English learners.
From a methodology aspect, relatively few intervention studies have used latent variable multilevel modeling. This method is useful for estimating treatment effects when considering the structure of data, as it allows a researcher to consider partially nested designs. In this case, students were nested in small groups in the experimental condition but were not nested in the control group. When a researcher ignores this complex nested structure, the standard error estimation is affected negatively, skewing the standardized effects upward. Further, the ML-SEM increases power to detect effects by using latent variables. The model minimizes measurement invariance inherent to using individual measures by estimating the common variance shared among tasks used to represent one construct, increasing reliability and reducing confidence intervals around an estimated effect size.
From an intervention aspect, this multicomponent intervention for reading comprehension provided significant effects for students in the experimental condition and showed positive effects specifically for students with limited English proficiency. The study was rigorous and implemented with high fidelity, representing one of the most comprehensive interventions of reading comprehension for upper-elementary students to date.
References
Mehta, P. D., & Neale, M. C. (2005). People are variables too: Multilevel structural equations modeling. Psychological Methods, 10, 259–284. doi:10.1037/1082-989X.10.3.259
Scammacca, N., Roberts, G., Vaughn, S., Edmonds, M., Wexler, J., Reutebuch, C. K., & Torgesen, J. K. (2007). Interventions for adolescent struggling readers: A meta-analysis with implications for practice. Portsmouth, NH: RMC Research, Center on Instruction.
Wanzek, J., Wexler, J., Vaughn, S., & Ciullo, S. (2010). Reading interventions for struggling readers in the upper elementary grades: A synthesis of 20 years of research. Reading and Writing, 23, 889–912. doi:10.1007/s11145-009-9179-5
What Works Clearinghouse. (2014). Procedures and standards handbook (Version 3.0). Washington, DC: U.S. Department of Education. Retrieved from https://ies.ed.gov/ncee/wwc/Docs/referenceresources/wwc_procedures_v3_0_standards_handbook.pdf