September 2019: A Meta-Analysis: Are Students With Disabilities Accessing the Curriculum?

Study Background

The academic performance of students with disabilities has continued to lag behind the performance of students without disabilities, despite decades of federal and state policy and legislation requiring students with disabilities to have full access to curriculum alongside their peers without disabilities. Over time, the definition of “access” has changed. One of the first pieces of legislation on access, Education for all Handicapped Children Act of 1975, focused on location, ensuring students with disabilities were identified and placed in neighborhood schools. However, it wasn’t until its reauthorization as the Individuals with Disabilities in Education Act Amendments of 1997 (IDEA) that students with disabilities would be expected to access and participate in state assessments. The concept of access was further modified with the No Child Left Behind Act of 2001 (NCLB), which required students with disabilities to not only participate in state assessments but also make progress, thus linking access to outcomes. In this way, the achievement gap has become a standard used to determine the extent to which students with disabilities have access to the general education curriculum. If students with disabilities have the support and services they need, we would expect the achievement gap to be minimal or nonexistent.

In reality, the achievement gap between students with and without disabilities is large (Albus, Lazarus & Thurlow, 2014; Wagner, Newman, Cameto, Levine, & Garza, 2006). In the area of reading, students with disabilities scored on average 1.39 standard deviations below a sample of typical readers in reading comprehension (Wagner et al., 2006). Additionally, students with disabilities enter school with lower reading achievement, and the gap persists over time (Morgan, Farkas, & Wu, 2011).

However, using achievement gaps to estimate curriculum access for students with disabilities has several limitations. First, when studies use a high proficiency cut point for measurement, the gap between students with disabilities and students without disabilities will be underestimated. Second, grouping students with disabilities into one category can hide differences in achievement that vary by disability type. Third, the use of achievement criteria based on typical readers to define achievement for students without disabilities in low-performing schools may overestimate the achievement gap, as most students, regardless of disability status, are performing poorly. Finally, researcher-collected data may use samples that are smaller and less representative than statewide assessments. As one can see, estimates of the achievement gap vary due to sample and assessment characteristics.

Study Purpose & Research Questions

With those limitations in mind, Gilmour, Fuchs, and Wehby (2018) conducted a meta-analysis of studies exploring the gap in reading achievement between students with and without disabilities to provide an accurate estimate of the achievement gap. Students with disabilities had an Individualized Education Program (IEP) or a disability specified in IDEA. The meta-analysis addressed the following research questions:

1. What is the average size of the reading achievement gap between school-age students with and without disabilities in the United States?
2. To what extent is the gap moderated by sample characteristics (type of disability, school level, and whether testing occurred before or after the No Child Left Behind Act [NCLB, 2001]) and assessment characteristics (high- or low-stakes tests and whether “reading” was defined as reading comprehension or as a composite of various reading skills)?

Methodology

Studies were included if they met the following criteria: (a) contained a sample of students with disabilities, defined as students with an IEP or an IDEA-specified disability; (b) contained a comparison group of students without disabilities, drawn from the same population (e.g., school, state); (c) the data were collected after 1997, to ensure continuity of disability labels under federal law; (d) students were in kindergarten through 12th grade in the United States; and (e) a measure of reading comprehension was used, with sufficient data to calculate effect sizes.

Studies were excluded if (a) the comparison group was created based on propensity score matching or matching on descriptive characteristics, (b) a proficiency cut score was used, or (c) samples were drawn from clinical settings.

Twenty three studies met inclusion criteria and 180 effect sizes were calculated from the studies.

Studies were coded for years of data collection, disability types, chronological age or grade, the level of data collection, how disability type was determined, and characteristics of excluded students. Study measures were coded for reading construct assessed, individual or group administration, high stakes or not, response types, retest or not, and whether accommodations were provided. Several variables were examined as potential moderators (e.g., disability types, age/grade).

Key Findings

• The average reading achievement gap between students with and without disabilities was 1.17 standard deviations, with students with disabilities underperforming students without disabilities.
• Looking at the category of learning disability (LD), students with LD performed 1.44 standard deviations below students without disabilities.
• The average achievement gap between students with disabilities and students without disabilities in elementary grades was 1.13 standard deviations. However, this gap was not statistically significant.
• The gaps between students with disabilities and students without disabilities were of similar magnitude before and after the implementation of NCLB, but these gaps were not statistically significant.

Limitations

The study investigated cross-sectional achievement gaps, instead of longitudinal gaps following cohorts of students, so inferences about the achievement gap over time cannot be made. As would be expected, there was a limited number of students with low-incidence disabilities included, and the alternate assessments these students often take were not included in this meta-analysis. This may result in an underestimation of the achievement gap for low incidence disabilities.

Implications for Practitioners

Given that 60% of fourth- and eighth-graders without disabilities are reading below grade level (NAEP, 2017), Gilmour and colleagues point out that the situation for students with disabilities is even more dire. Although the meta-analysis conducted by Gilmour and colleagues does not identify the causes of this achievement gap, they do hypothesize that the magnitude of the gap is related to access. At the same time, the results of the meta-analysis prompt the authors to reconsider the feasibility of closing the achievement gap. Gilmour and colleagues present the following questions and discussion points:

Why is the achievement gap so large?

• Misunderstanding of the least restrictive environment (LRE): The general education classroom, even with accommodations, may not support the delivery of intensive services to students with disabilities with the most persistent reading needs. Equating setting with access is a misunderstanding of LRE, as exposure to curriculum alone will not result in success in reading for students with disabilities. For some students with disabilities, access to the general education curriculum is not enough; they will need instruction beyond the general education classroom to close the achievement gap.
• Lack of evidence-based practices in reading instruction: Reading instruction for students with disabilities still may not include the instructional practices supported by research. In fact, reviews of observational studies have confirmed that instruction for students with or at risk of disabilities does not address foundational reading skills (Lindstrom, 2018). Teachers should be aware of and use instructional practices supported by research.

Is closing the achievement gap realistic?

• Contextualizing the achievement gap: Even with the strongest reading instruction, some students with disabilities may lag behind their peers. The focus on closing achievement gaps may obscure the reality that students with disabilities receive special education services specifically because of underperformance. The focus should perhaps be redirected to reducing the size of the achievement gap. For teachers, the focus should remain squarely on providing intensive, individualized instruction to students with disabilities and ensuring that these students make adequate progress.

How can policy close the gap?

• Increased emphasis on school implementation of service delivery: Many barriers exist for schools to actualize ideal models of service delivery to students with disabilities (e.g., recruitment and retention of high-quality teachers). From a policy perspective, state and federal educational agencies should consider the use and sustainment of effective service delivery models, such as multitiered systems of support. Even if teachers identify and have knowledge of the evidence-based practices referred to above, structures and supports must be in place to facilitate delivery of instruction to students with disabilities.

References

Albus, D., Lazarus, S. S., & Thurlow, M. L. (2014). 2011–12 publicly reported assessment results for students with disabilities and ELLs with disabilities (Tech. Rep. No. 69). Minneapolis, MN: National Center on Educational Outcomes.

Gilmour, A. F., Fuchs, D., & Wehby, J. H. (2019). Are students with disabilities accessing the curriculum? A meta-analysis of the reading achievement gap between students with and without disabilities. Exceptional Children, 85(3), 329–346.

Hill, C. J., Bloom, H. S., Black, A. R., & Lipsey, M. W. (2008). Empirical benchmarks for Interpreting effect sizes in research. Child Development Perspectives, 2, 172–177. doi:10.1111/j.1750-8606.2008.00061.x

Lindström, E. (2018). Reading instruction for students with and at-risk for disabilities: A systematic review of observation studies. Working paper.

Morgan, P. L., Farkas, G., & Wu, Q. (2011). Kindergarten children’s growth trajectories in reading and mathematics: Who falls increasingly behind? Journal of Learning Disabilities, 44, 472–88. doi:10.1177/0022219411414010

No Child Left Behind Act of 2001, Pub. L. No. 107-110, §115, Stat. 1425 (2002). Retrieved from www2.ed.gov/policy/elsec/leg/esea02/107-110.pdf

Wagner, M., Newman, L., Cameto, R., Levine, P., & Garza, N. (2006). An overview of findings from Wave 2 of the National Longitudinal Transition Study-2 (NLTS2) (NCSER 2006–3004). Washington, DC: National Center for Special Education Research.